Home » Software Development » Anthos ASM: Cross cluster mesh with different network

About Rajeev Hathi

Rajeev Hathi
Rajeev is a senior Java architect and developer. He has been designing and developing business applications for various companies (both product and services). He is co-author of the book titled 'Apache CXF Web Service Development' and shares his technical knowledge through his blog platform techorgan.com

Anthos ASM: Cross cluster mesh with different network

The previous article demonstrated how to implement cross-cluster Anthos Service Mesh (ASM) in a single network. In this article we will illustrate the mesh service communication using different VPC network. The use case will focus on setting up two GKE cluster with ASM installed in two different VPCs. As the clusters are in different networks, the services in a mesh will communicate with each other via internal east-west ingress gateway. The communication will happen over the Internet.

The mesh topology will look like the following:

Prerequisites

  • You have already setup a Google project with two VPCs in separate regions. You can opt for any available two regions to implement this use case. For this article we will take Mumbai as one region and Singapore as another.
  • You have Anthos GKE cluster setup in both the VPCs along with ASM version 1.8.3 installed. To understand how to install ASM, follow this link. You will also have to install istio operator that will represent ingress gateway to serve the east-west traffic across two clusters. See section Set up east-west gateway below.
  • The ASM Certificate Authority (CA) used will be Mesh CA (only available for GKE clusters). You could also use Citadel CA as an alternate option.

Set the cluster context

As a first step, identify the context of each cluster. The below command will lists the different cluster context.

1
2
3
kubectl config get-contexts -o name
gke_sandbox-111111_asia_south1-a_cluster-1
gke_sandbox-111111_asia_southeast1-a_cluster-2

The cluster context name follows a pattern: project-id_cluster-location_cluster_name. Assign the context name to $ctx1 and $ctx2 environment variables, each representing cluster one and two respectively.

1
export ctx1=gke_sandbox-111111_asia_south1-a_cluster-1 export ctx2=gke_sandbox-111111_asia_southeast1-a_cluster-2

Setup endpoint discovery between clusters

In this step you will enable each cluster to discover service endpoints of their counterpart, so that cluster one will discover service endpoints of the second cluster and vice versa.

1
istioctl x create-remote-secret --context=$ctx1 --name=cluster-1 | \ kubectl apply -f - --context=$ctx2 istioctl x create-remote-secret --context=$ctx2 --name=cluster-2 | \ kubectl apply -f - --context=$ctx1

You enable this by creating secrets for each cluster that grants access to kube API server of that cluster. Each secret is the certificate derived from the common root CA, in this case Mesh CA. You then apply the secret to the other cluster. In that way secrets are exchanged and the clusters are able to see the service endpoints of each other. In this case the endpoint of ingress gateway that will serve the east-west traffic will be discovered.

Set up east-west gateway

Before installing ASM, create an istio operator config file in both the clusters which will set up an ingress gateway for east-west traffic (cross-cluster). The Mumbai cluster will represent west side of the traffic while the Singapore cluster will represent east side of the traffic. The config file for the Mumbai cluster will look like the following:

01
02
03
04
05
06
07
08
09
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
apiVersion: install.istio.io/v1alpha1
 kind: IstioOperator
 spec:
   values:
     global:
       meshID: mesh1
       multiCluster:
         clusterName: cluster-1
       network: vpc-1
   components:
     ingressGateways:
       - name: istio-eastwestgateway
         label:
           istio: eastwestgateway
           app: istio-eastwestgateway
           topology.istio.io/network: vpc-1
         enabled: true
         k8s:
           env:
             - name: ISTIO_META_ROUTER_MODE
               value: "sni-dnat"
             - name: ISTIO_META_REQUESTED_NETWORK_VIEW
               value: vpc-1
           service:
             ports:
               - name: status-port
                 port: 15021
                 targetPort: 15021
               - name: tls
                 port: 15443
                 targetPort: 15443
               - name: tls-istiod
                 port: 15012
                 targetPort: 15012
               - name: tls-webhook
                 port: 15017
                 targetPort: 15017

You can apply the above configuration as an overlay file while installing ASM using the -f flag

1
istioctl install --context=$ctx1 -f istio-ingress-eastwest.yaml

Important thing to note is the port 15443 which is used for cross cluster communication. In order to expose this port for mesh services to communicate, you will have to create a custom gateway that will use Service Name Indication (SNI) routing with TLS extension on this port. The below code depicts the custom gateway that will actually facilitate east-west routing.

01
02
03
04
05
06
07
08
09
10
11
12
13
14
15
16
17
apiVersion: networking.istio.io/v1alpha3
 kind: Gateway
 metadata:
   name: multi-network
   namespace: istio-system
 spec:
   selector:
     istio: eastwestgateway
   servers:
     - port:
         number: 15443
         name: tls
         protocol: TLS
       tls:
         mode: AUTO_PASSTHROUGH
       hosts:
         - "*.local"

The above gateway named multi-network exposes port 15443 using SNI (with TLS extension) based routing allowing Kubernetes services with .local domain to communicate with Mumbai cluster mesh via this gateway. The TLS mode used is AUTO_PASSTHROUGH. This will, in effect, not terminate the TLS connection at the gateway and instead pass on the request to the target backend service. This mode also assumes that the client or the service from the calling cluster has initiated a TLS handshake. So if the service from the Singapore cluster uses plain vanilla http:// and not https://, the gateway will still pass through the request. This can create security loophole if the target backend does not handle TLS termination. We have kept it simple, as the purpose of this use case is to simply demonstrate cross cluster communication between two VPCs.

Apply both the above config file in the Singapore cluster (just change the name of the network) and you should be all set. To setup the sample application and test the service to service communication across clusters you can refer to the previous article.

Published on Java Code Geeks with permission by Rajeev Hathi, partner at our JCG program. See the original article here: Anthos ASM: Cross cluster mesh with different network

Opinions expressed by Java Code Geeks contributors are their own.

(0 rating, 0 votes)
You need to be a registered member to rate this.
Start the discussion Views Tweet it!
Do you want to know how to develop your skillset to become a Java Rockstar?
Subscribe to our newsletter to start Rocking right now!
To get you started we give you our best selling eBooks for FREE!
1. JPA Mini Book
2. JVM Troubleshooting Guide
3. JUnit Tutorial for Unit Testing
4. Java Annotations Tutorial
5. Java Interview Questions
6. Spring Interview Questions
7. Android UI Design
and many more ....
I agree to the Terms and Privacy Policy
Subscribe
Notify of
guest

This site uses Akismet to reduce spam. Learn how your comment data is processed.

0 Comments
Inline Feedbacks
View all comments