In this blog article I will show you how to realize transparent user authentication for an authentication-unaware application using Istio and Azure Active Directory.
Istio has many features, some of which are not very well documented. One of those features that peaked my interest, when working on bringing Istio to production, is authentication and authorization of end-users based on JSON Web Tokens (JWTs). In this blog post I will explore what exactly Istio has to offer for end-user authentication, what you must build yourself and how to do it. I will use Azure Active Directory (Azure AD) as Identity Provider, but the same principles can also be applied to other Authentication Providers.
A short introduction to Istio
You can safely skip this part if you already have experience with Istio. I will also only focus on the parts relevant to this blog article, for a more comprehensive overview of Istio refer to the official documentation.
Istio is a Service Mesh, meaning that it solves common application features related to networking outside the application code. Istio does that by adding a sidecar proxy to each instance of an application, usually a Kubernetes pod, and orchestrating these proxies from a central control plane.
This allows Istio, among other things, to transparently encrypt all traffic with mTLS and apply authorization policies to all services centrally. Instead of Ingress, Istio uses Gateways to serve as an entrypoint into the mesh where different ways of authentication can be used.
Other features that are not relevant to this article (but still cool!) are advanced traffic management features, such as canary releases or support for the circuit breaker pattern.
User authentication with Istio
At first, my motivation for this blog entry was out of curiosity. The Istio security documentation describes a feature called Request authentication:
“Used for end-user authentication to verify the credential attached to the request. Istio enables request-level authentication with JSON Web Token (JWT) validation and a streamlined developer experience using a custom authentication provider or any OpenID Connect providers […].“
In my mind, that sounded like authentication would just be a simple switch that needs to be flipped with some Istio Custom Resource, as I have often experienced with Istio.
The technical details with precise instructions can be found in my demo repository on GitHub.
To test this, I started a small Azure Kubernetes Service (AKS) Cluster and installed Istio in demo mode using istioctl. I also created a DNS entry for my test application, generated a certificate using letsencrypt and set up the Istio Ingress Gateway with this certificate. I chose Prometheus as an authentication-unaware service to enhance, because it was already part of the Istio demo-install and is actually an application that does not implement authentication.
For authentication I created an Azure app registration for the public URL where my prometheus resides and exposed it as web API. That, amongst other things, causes an OAuth2 Permission Scope to be generated.
Now that the basic setup stands, I need to do three things:
- obtain an access_token
- validate the token
- ensure the token is passed on subsequent requests
Obtaining a JWT is easy enough using a VirtualService for redirecting users to login.microsoft.com’s OAuth2 login for our Azure Tenant giving the App Scope that I generated before:
|
1 2 3 4 5 6 7 8 9 10 11 |
apiVersion: networking.istio.io/v1alpha3 kind: VirtualService metadata: name: prometheus spec: ... http: - redirect: uri: /<azureTenantId>/oauth2/v2.0/authorize?client_id=<azureAppId>&response_type=code&redirect_uri=https%3A%2F%2F<our-url>%2Floggedin&response_mode=form_post&scope=<azure-app-scope> authority: login.microsoftonline.com redirectCode: 302 |
Validating the token can then be done using Istio’s RequestAuthentication Custom Resource, again providing the correct URIs for our tenant and App:
|
1 2 3 4 5 6 7 8 9 10 11 |
apiVersion: security.istio.io/v1beta1 kind: RequestAuthentication metadata: name: ingressgateway spec: selector: matchLabels: istio: ingressgateway jwtRules: - issuer: "https://sts.windows.net/<azureTenantId>/" jwksUri: "https://login.microsoftonline.com/<azureTenantId>/discovery/keys?appid=<azureAppId>" |
This Custom Resource alone only attaches an authenticated identity to requests that have a valid JWT, though. To also authorize users, I need another Resource called AuthorizationPolicy. The AuthorizationPolicy is configured to allow authenticated users access everywhere, which implicitly denies unauthenticated users access (as no ALLOW rule matches them).
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 |
apiVersion: security.istio.io/v1beta1 kind: AuthorizationPolicy metadata: name: ingressgateway spec: selector: matchLabels: istio: ingressgateway action: ALLOW rules: # this rule allows all authenticated users access - from: - source: requestPrincipals: ["*"] |
With this rule setup, I now only need to ensure that every subsequent request to Prometheus passes its users bearer-token in the Authorization header. This is actually not as easy as it sounds, because there is no mechanism for telling our user to set this header without implementing it in our authentication-unaware application. Because the goal was not to modify Prometheus, I instead implemented a helper application that accepts the login response from Azure and sets the token as a cookie. The cookie will be passed by the user automatically but Istio does not directly support it. The RequestAuthentication could theoretically be configured to look in the cookie header for the token, but when multiple cookies would be set, it would not be able to distinguish the token from the other cookies.
As a workaround, I chose to simply rewrite the cookie contents into the Authorization header using an EnvoyFilter. This filter executes a Lua script before the JWT authentication step, that checks if the cookie is present and sets the Authorization header if it is. If no cookie is present, it redirects the user to the /login endpoint on the original address. Requests to /login are then exempt from the AuthorizationPolicy, so that the original redirect in our VirtualService can be applied.
Additionally, I set up a /loggedin route in our VirtualService that redirects to the helper application and also exempt it from authorization. Finally, I omitted one detail in the initial VirtualService: It of course also has to redirect to Prometheus when a user is authenticated. With the cookie approach, this is done with following route:
|
1 2 3 4 5 6 7 8 9 10 11 |
http: - name: authenticated-users match: - headers: cookie: regex: ".*access_token=.*" route: - destination: port: number: 9090 host: prometheus |
With this setup, only authorized users can now reach Prometheus, achieving our goal of transparent (to the application) user authentication using Istio.
Drawbacks of my approach
One issue I found with this, is that the user authentication can only be applied to the Istio Ingress Gateway, meaning that in a production environment separate gateway deployments should probably be used for authenticated and unauthenticated endpoints. My workaround makes this issue more dire, as the Lua script cannot distinguish between endpoints that should be authenticated and endpoints that should not.
My demo is also lacking a logout mechanism. In a productive setup a proper tool should be used for handling the authentication callbacks, e.g. OAuth2 Proxy. Adding support to Istio for retrieving the JWT from cookies would also simplify the whole process immensely.
Comparison with other approaches
When Istio is not already part of your setup, you probably should not install it just for user authentication. Instead two patterns have been used in previous projects of mine, that can enhance an application with authentication on the infrastructure layer in Kubernetes.
The first is using a sidecar container in your application pods running for example keycloak-gatekeeper . This means, each of your pods will grow in size and your requests will pass through the sidecar, which poses a similar overhead as Istio (but for less gain). Edit: keycloak-gatekeeper has been sunsetted in favor of oauth2-proxy since my research for this article. The approach is still valid, since OAuth2-Proxy can be deployed as sidecar.
The second alternative is securing your application at the Ingress level using the OAuth2-Proxy integration of ingress-nginx. Here, for each request received by nginx, an authentication subrequest is made to OAuth2-Proxy, which supports many cloud providers and Open Source SSO solutions natively. The overhead is lower than with the sidecar approach, but you still need one OAuth2-Proxy deployment per OAuth2 client, as multiple configs are not supported.
While both approaches allow restricting logins to some groups (depending on the used provider in case of OAuth2-Proxy), they still are not as flexible and powerful as Istio.
Closing remarks
Enabling user authentication was not as easy as I initially thought. To be fair, the documentation that sent me on my journey only talks about authentication for attached credentials so I have expected more than it was promised. I still think that it would be nice to have a simple option to “switch on“ authentication, maybe through integration with OAuth2-Proxy similar to how ingress-nginx solved it. But you can already use Request Authentication in Istio to transparently authenticate user requests with some extra work, and I think it is really neat.
