This guide demonstrates a client within the service mesh accessing destinations external to the mesh using OSM’s Egress policy API.

Prerequisites

  • Kubernetes cluster running Kubernetes v1.19.0 or greater.
  • Have OSM installed.
  • Have kubectl available to interact with the API server.
  • Have osm CLI available for managing the service mesh.

Demo

  1. Enable egress policy if not enabled:

    export osm_namespace=<osm-namespace> # Replace <osm-namespace> with the namespace where OSM is installed
    kubectl patch meshconfig osm-mesh-config -n "$osm_namespace" -p '{"spec":{"featureFlags":{"enableEgressPolicy":true}}}'  --type=merge
    
  2. Deploy the curl client into the curl namespace after enrolling its namespace to the mesh.

    # Create the curl namespace
    kubectl create namespace curl
    
    # Add the namespace to the mesh
    osm namespace add curl
    
    # Deploy curl client in the curl namespace
    kubectl apply -f https://raw.githubusercontent.com/openservicemesh/osm/main/docs/example/manifests/samples/curl/curl.yaml -n curl
    

    Confirm the curl client pod is up and running.

    $ kubectl get pods -n curl
    NAME                    READY   STATUS    RESTARTS   AGE
    curl-54ccc6954c-9rlvp   2/2     Running   0          20s
    

HTTP Egress

  1. Confirm the curl client is unable make the HTTP request http://httpbin.org:80/get to the httpbin.org website on port 80.

    $ kubectl exec $(kubectl get pod -n curl -l app=curl -o jsonpath='{.items..metadata.name}') -n curl -c curl -- curl -sI http://httpbin.org:80/get
    command terminated with exit code 7
    
  2. Apply an Egress policy to allow the curl client’s ServiceAccount to access the httpbin.org website on port 80 serving the http protocol.

    kubectl apply -f - <<EOF
    kind: Egress
    apiVersion: policy.openservicemesh.io/v1alpha1
    metadata:
      name: httpbin-80
      namespace: curl
    spec:
      sources:
      - kind: ServiceAccount
        name: curl
        namespace: curl
      hosts:
      - httpbin.org
      ports:
      - number: 80
        protocol: http
    EOF
    
  3. Confirm the curl client is able to make successful HTTP requests to http://httpbin.org:80/get.

    $ kubectl exec $(kubectl get pod -n curl -l app=curl -o jsonpath='{.items..metadata.name}') -n curl -c curl -- curl -sI http://httpbin.org:80/get
    HTTP/1.1 200 OK
    date: Thu, 13 May 2021 21:49:35 GMT
    content-type: application/json
    content-length: 335
    server: envoy
    access-control-allow-origin: *
    access-control-allow-credentials: true
    x-envoy-upstream-service-time: 168
    
  4. Confirm the curl client can no longer make successful HTTP requests to http://httpbin.org:80/get when the above policy is removed.

    kubectl delete egress httpbin-80 -n curl
    
    $ kubectl exec $(kubectl get pod -n curl -l app=curl -o jsonpath='{.items..metadata.name}') -n curl -c curl -- curl -sI http://httpbin.org:80/get
    command terminated with exit code 7
    

HTTPS Egress

Since HTTPS traffic is encrypted with TLS, OSM routes HTTPS based traffic by proxying it to its original destination as a TCP stream. The Server Name Indication (SNI) indicated by the HTTPS client application in the TLS handshake is matched against hosts specified in the Egress policy.

  1. Confirm the curl client is unable make the HTTPS request https://httpbin.org:443/get to the httpbin.org website on port 443.

    $ kubectl exec $(kubectl get pod -n curl -l app=curl -o jsonpath='{.items..metadata.name}') -n curl -c curl -- curl -sI https://httpbin.org:443/get
    command terminated with exit code 7
    
  2. Apply an Egress policy to allow the curl client’s ServiceAccount to access the httpbin.org website on port 443 serving the https protocol.

    kubectl apply -f - <<EOF
    kind: Egress
    apiVersion: policy.openservicemesh.io/v1alpha1
    metadata:
      name: httpbin-443
      namespace: curl
    spec:
      sources:
      - kind: ServiceAccount
        name: curl
        namespace: curl
      hosts:
      - httpbin.org
      ports:
      - number: 443
        protocol: https
    EOF
    
  3. Confirm the curl client is able to make successful HTTPS requests to https://httpbin.org:443/get.

    $ kubectl exec $(kubectl get pod -n curl -l app=curl -o jsonpath='{.items..metadata.name}') -n curl -c curl -- curl -sI https://httpbin.org:443/get
    HTTP/2 200
    date: Thu, 13 May 2021 22:09:36 GMT
    content-type: application/json
    content-length: 260
    server: gunicorn/19.9.0
    access-control-allow-origin: *
    access-control-allow-credentials: true
    
  4. Confirm the curl client can no longer make successful HTTPS requests to https://httpbin.org:443/get when the above policy is removed.

    kubectl delete egress httpbin-443 -n curl
    
    $ kubectl exec $(kubectl get pod -n curl -l app=curl -o jsonpath='{.items..metadata.name}') -n curl -c curl -- curl -sI https://httpbin.org:443/get
    command terminated with exit code 7
    

TCP Egress

TCP based Egress traffic is matched against the destination port and IP address ranges specified in an egress policy. If an IP address range is not specified, traffic will be matched only based on the destination port.

  1. Confirm the curl client is unable make the HTTPS request https://openservicemesh.io:443 to the openservicemesh.io website on port 443. Since HTTPS uses TCP as the underlying transport protocol, TCP based routing should implicitly enable access to any HTTP(s) host on the specified port.

    $ kubectl exec $(kubectl get pod -n curl -l app=curl -o jsonpath='{.items..metadata.name}') -n curl -c curl -- curl -sI https://openservicemesh.io:443
    command terminated with exit code 7
    
  2. Apply an Egress policy to allow the curl client’s ServiceAccount to access the any destination on port 443 serving the tcp protocol.

    kubectl apply -f - <<EOF
    kind: Egress
    apiVersion: policy.openservicemesh.io/v1alpha1
    metadata:
      name: tcp-443
      namespace: curl
    spec:
      sources:
      - kind: ServiceAccount
        name: curl
        namespace: curl
      ports:
      - number: 443
        protocol: tcp
    EOF
    

    Note: For server-first protocols such as MySQL, PostgreSQL, etc., where the server initiates the first bytes of data between the client and server, the protocol must be set to tcp-server-first to indicate to OSM to not perform protocol detection on the port. Protocol detection relies on inspecting the initial bytes of a connection, which is incompatible with server-first protocols. When the port’s protocol is set to tcp-server-first, protocol detection is skipped for that port number. It is also important to note that server-first port numbers must not be used for other application ports that require protocol detection to performed, which means the port numbers used for server-first protocols must not be used with other protocols such as HTTP and TCP that require protocol detection to be performed.

  3. Confirm the curl client is able to make successful HTTPS requests to https://openservicemesh.io:443.

    $ kubectl exec $(kubectl get pod -n curl -l app=curl -o jsonpath='{.items..metadata.name}') -n curl -c curl -- curl -sI https://openservicemesh.io:443
    HTTP/2 200
    cache-control: public, max-age=0, must-revalidate
    content-length: 0
    content-type: text/html; charset=UTF-8
    date: Thu, 13 May 2021 22:35:07 GMT
    etag: "353ebaaf9573718bd1df6b817a472e47-ssl"
    strict-transport-security: max-age=31536000
    age: 0
    server: Netlify
    x-nf-request-id: 35a4f2dc-5356-45dc-9208-63e6fa162e0f-3350874
    
  4. Confirm the curl client can no longer make successful HTTPS requests to https://openservicemesh.io:443 when the above policy is removed.

    kubectl delete egress tcp-443 -n curl
    
    $ kubectl exec $(kubectl get pod -n curl -l app=curl -o jsonpath='{.items..metadata.name}') -n curl -c curl -- curl -sI https://openservicemesh.io:443
    command terminated with exit code 7
    

HTTP Egress with SMI route matches

HTTP Egress policies can specify SMI HTTPRouteGroup matches for fine grained traffic control based on HTTP methods, headers and paths.

  1. Confirm the curl client is unable make HTTP requests to http://httpbin.org:80/get and http://httpbin.org:80/status/200 to the httpbin.org website on port 80.

    $ kubectl exec $(kubectl get pod -n curl -l app=curl -o jsonpath='{.items..metadata.name}') -n curl -c curl -- curl -sI http://httpbin.org:80/get
    command terminated with exit code 7
    $ kubectl exec $(kubectl get pod -n curl -l app=curl -o jsonpath='{.items..metadata.name}') -n curl -c curl -- curl -sI http://httpbin.org:80/status/200
    command terminated with exit code 7
    
  2. Apply an SMI HTTPRouteGroup resource to allow access to the HTTP path /get and an Egress policy to access the httpbin.org on website port 80 that matches on the SMI HTTPRouteGroup.

    kubectl apply -f - <<EOF
    apiVersion: specs.smi-spec.io/v1alpha4
    kind: HTTPRouteGroup
    metadata:
      name: egress-http-route
      namespace: curl
    spec:
      matches:
      - name: get
        pathRegex: /get
    ---
    kind: Egress
    apiVersion: policy.openservicemesh.io/v1alpha1
    metadata:
      name: httpbin-80
      namespace: curl
    spec:
      sources:
      - kind: ServiceAccount
        name: curl
        namespace: curl
      hosts:
      - httpbin.org
      ports:
      - number: 80
        protocol: http
      matches:
      - apiGroup: specs.smi-spec.io/v1alpha4
        kind: HTTPRouteGroup
        name: egress-http-route
    EOF
    
  3. Confirm the curl client is able to make successful HTTP requests to http://httpbin.org:80/get.

    $ kubectl exec $(kubectl get pod -n curl -l app=curl -o jsonpath='{.items..metadata.name}') -n curl -c curl -- curl -sI http://httpbin.org:80/get
    HTTP/1.1 200 OK
    date: Thu, 13 May 2021 21:49:35 GMT
    content-type: application/json
    content-length: 335
    server: envoy
    access-control-allow-origin: *
    access-control-allow-credentials: true
    x-envoy-upstream-service-time: 168
    
  4. Confirm the curl client is unable to make successful HTTP requests to http://httpbin.org:80/status/200.

    $ kubectl exec $(kubectl get pod -n curl -l app=curl -o jsonpath='{.items..metadata.name}') -n curl -c curl -- curl -sI http://httpbin.org:80/status/200
    HTTP/1.1 404 Not Found
    date: Fri, 14 May 2021 17:08:48 GMT
    server: envoy
    transfer-encoding: chunked
    
  5. Update the matching SMI HTTPRouteGroup resource to allow requests to HTTP paths matching the regex /status.*.

    kubectl apply -f - <<EOF
    apiVersion: specs.smi-spec.io/v1alpha4
    kind: HTTPRouteGroup
    metadata:
      name: egress-http-route
      namespace: curl
    spec:
      matches:
      - name: get
        pathRegex: /get
      - name: status
        pathRegex: /status.*
    EOF
    
  6. Confirm the curl client can now make successful HTTP requests to http://httpbin.org:80/status/200.

    $ kubectl exec $(kubectl get pod -n curl -l app=curl -o jsonpath='{.items..metadata.name}') -n curl -c curl -- curl -sI http://httpbin.org:80/status/200
    HTTP/1.1 200 OK
    date: Fri, 14 May 2021 17:10:48 GMT
    content-type: text/html; charset=utf-8
    content-length: 0
    server: envoy
    access-control-allow-origin: *
    access-control-allow-credentials: true
    x-envoy-upstream-service-time: 188