gRPC 服務通過 Istio 網格通信
引言
本文通過gRCP服務消費方mesha和gRPC服務提供方meshb,驗證其部署在Istio網格的通信過程。通過該示例可以將外部注冊中心接入網格,不再困難。
一、Isito配置提點
在Kubernetes集群中已安裝了Isito,并查驗下面幾個參數是否正確設置。
istio-config.yaml內容
- ---
- apiVersion: install.istio.io/v1alpha1
- kind: IstioOperator
- spec:
- profile: default
- values:
- global:
- logging:
- level: default:debug
- meshConfig:
- accessLogFile: /dev/stdout
- defaultConfig:
- holdApplicationUntilProxyStarts: true
- proxyMetadata:
- ISTIO_META_DNS_CAPTURE: "true"
- ISTIO_META_DNS_AUTO_ALLOCATE: "true"
- components:
- pilot:
- hub: istio
- tag: 1.10.4
關鍵參數說明
參數 | 說明 |
---|---|
holdApplicationUntilProxyStarts | 默認false,設置為true表示業(yè)務容器需在sidecar Proxy容器啟動完畢后啟動 |
ISTIO_META_DNS_CAPTURE | 默認false,設置為true表示開啟DNS代理,DNS的請求將被轉發(fā)到sidecar |
ISTIO_META_DNS_AUTO_ALLOCATE | 默認false,設置為true表示DNS代理將自動為ServiceEntrys分配IP,不需要指定 |
執(zhí)行如下命令生效配置
- istioctl install -y -f istio-config.yaml
通過如下命令驗證生效后的配置
- kubectl describe IstioOperator installed-state -n istio-system >> istioOperator.conf
二、示例說明
定義Proto
通過簡單方法SayHello完成client與server通信。
- syntax = "proto3";
- option java_multiple_files = true;
- option java_package = "com.melon.test.client.grpc";
- option java_outer_classname = "HelloMesh";
- package meshgrpc;
- service Mesher {
- rpc SayHello (HelloRequest) returns (HelloReply) {}
- }
- message HelloRequest {
- string name = 1;
- }
- message HelloReply {
- string message = 1;
- }
服務消費方mesha
- @RestController
- public class MeshSender {
- @GetMapping("demo")
- public String meshWorker(){
- String target = "dns:///AppMeshClient.mesh:50000";
- // String target = "127.0.0.1:50000";
- ManagedChannel channel = ManagedChannelBuilder.forTarget(target)
- .usePlaintext()
- .build();
- MesherGrpc.MesherBlockingStub blockingStub = MesherGrpc.newBlockingStub(channel);
- HelloRequest request = HelloRequest.newBuilder().setName("mesh demo!").build();
- HelloReply reply = blockingStub.sayHello(request);
- System.out.println(reply.getMessage());
- return reply.getMessage();
- }
- }
服務提供方meshb
- @Component
- public class MeshReceiver {
- @PostConstruct
- public void receiverWorker() throws IOException {
- int port = 50000;
- Server server = ServerBuilder.forPort(port)
- .addService(new MeshBService())
- .build()
- .start();
- System.out.println("Server started.");
- }
- class MeshBService extends MesherGrpc.MesherImplBase {
- public void sayHello(HelloRequest request,
- io.grpc.stub.StreamObserver<HelloReply> responseObserver) {
- System.out.println("receiver client message: " + request.getName());
- HelloReply reply = HelloReply.newBuilder().setMessage("I'm from server " + request.getName()).build();
- responseObserver.onNext(reply);
- responseObserver.onCompleted();
- }
- }
- }
鏡像推送
- <plugin>
- <groupId>com.google.cloud.tools</groupId>
- <artifactId>jib-maven-plugin</artifactId>
- <version>3.1.4</version>
- <configuration>
- <from>
- <image>
- harbor.x.x/x/java:8u212-full
- </image>
- <auth>
- <username>${harbor.username}</username>
- <password>${harbor.password}</password>
- </auth>
- </from>
- <to>
- <image>x.x.x/x/${project.name}</image>
- <auth>
- <username>${harbor.username}</username>
- <password>${harbor.password}</password>
- </auth>
- <tags>
- <tag>${project.version}</tag>
- <tag>latest</tag>
- </tags>
- </to>
- <container>
- <ports>
- <port>x</port>
- </ports>
- <creationTime>USE_CURRENT_TIMESTAMP</creationTime>
- </container>
- </configuration>
- </plugin>
說明:通過maven插件jib,執(zhí)行命令如下命令「mvn compile jib:build」將鏡像推到harbor倉庫
三、gRPC服務提供方部署
設置meshb服務的Deployment
- ---
- apiVersion: apps/v1
- kind: Deployment
- metadata:
- name: meshb
- labels:
- app: meshb
- spec:
- selector:
- matchLabels:
- app: meshb
- replicas: 1
- template:
- metadata:
- labels:
- app: meshb
- spec:
- imagePullSecrets:
- - name: xxxx #倉庫名稱
- containers:
- - name: meshb
- image: x.x.x.x/x/meshb:latest
- imagePullPolicy: Always
- ports:
- - containerPort: 50000
執(zhí)行下面命令生效
- kubectl apply -f meshb.yaml
- deployment.apps/meshb created
查看運行狀態(tài)正常
- # kubectl get pods
- NAME READY STATUS RESTARTS AGE
- meshb-565945d794-wcb8z 2/2 Running 0 9m19s
查看啟動日志啟動成功
- # kubectl logs meshb-565945d794-wcb8z -n default
- Server started.
- 2021-11-05 09:21:19.828 INFO 1 --- [ main] o.s.b.w.embedded.tomcat.TomcatWebServer : Tomcat started on port(s): 8181 (http) with context path ''
- 2021-11-05 09:21:19.847 INFO 1 --- [ main] com.melon.test.MeshbApplication : Started MeshbApplication in 3.859 seconds (JVM running for 4.344)
備注:至此gRPC服務提供方已部署完成。
四、gRPC服務消費方部署
設置mesha服務的Deployment
- ---
- apiVersion: apps/v1
- kind: Deployment
- metadata:
- name: mesha
- labels:
- app: mesha
- spec:
- selector:
- matchLabels:
- app: mesha
- replicas: 1
- template:
- metadata:
- labels:
- app: mesha
- spec:
- imagePullSecrets:
- - name: middleware
- containers:
- - name: mesha
- image: harbor.hellobike.cn/base/mesha:latest
- ports:
- - containerPort: 7171
- imagePullPolicy: Always
- ---
- apiVersion: v1
- kind: Service
- metadata:
- name: mesha
- spec:
- selector:
- app: mesha
- type: LoadBalancer
- ports:
- - name: web
執(zhí)行命令部署
- # kubectl apply -f mesha.yaml
- deployment.apps/mesha created
- service/mesha created
查看服務狀態(tài)
- # kubectl get service
- NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
- mesha LoadBalancer 10.x.61.x <pending> 7171:30514/TCP 20s
查看Pod運行狀態(tài)
- # kubectl get pods
- NAME READY STATUS RESTARTS AGE
- mesha-b559fc4f4-m9752 2/2 Running 0 87s
備注:至此服務消費方已部署完成。
四、服務調用驗證
通過域名訪問
- http://x.x.x.x:30514/demo
頁面打印如下錯誤
查看日志發(fā)現域名無法解析:
- kubectl logs -f mesha-b559fc4f4-m9752 -n default
- 2021-11-05 09:01:37.372 WARN 1 --- [ault-executor-0] io.grpc.internal.ManagedChannelImpl : [Channel<1>: (dns:///AppMeshClient.mesh:50000)] Failed to resolve name. status=Status{code=UNAVAILABLE, description=Unable to resolve host AppMeshClient.mesh, cause=java.lang.RuntimeException: java.net.UnknownHostException: AppMeshClient.mesh: Name or service not known
- at io.grpc.internal.DnsNameResolver.resolveAll(DnsNameResolver.java:436)
- at io.grpc.internal.DnsNameResolver$Resolve.resolveInternal(DnsNameResolver.java:272)
- at io.grpc.internal.DnsNameResolver$Resolve.run(DnsNameResolver.java:228)
- at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
- at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
- at java.lang.Thread.run(Thread.java:748)
- Caused by: java.net.UnknownHostException: AppMeshClient.mesh: Name or service not known
- at java.net.Inet4AddressImpl.lookupAllHostAddr(Native Method)
- at java.net.InetAddress$2.lookupAllHostAddr(InetAddress.java:929)
- at java.net.InetAddress.getAddressesFromNameService(InetAddress.java:1324)
- at java.net.InetAddress.getAllByName0(InetAddress.java:1277)
- at java.net.InetAddress.getAllByName(InetAddress.java:1193)
- at java.net.InetAddress.getAllByName(InetAddress.java:1127)
- at io.grpc.internal.DnsNameResolver$JdkAddressResolver.resolveAddress(DnsNameResolver.java:646)
- at io.grpc.internal.DnsNameResolver.resolveAll(DnsNameResolver.java:404)
- ... 5 more
- }
通過ServiceEntry映射服務提供方IP
查看服務提供方meshb的IP為x.x.0.17
- # kubectl get pods -o wide
- NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
- mesha-b559fc4f4-m9752 2/2 Running 0 140m x.x.1.117 k8s-servicemesh-3 <none> <none>
- meshb-565945d794-wcb8z 2/2 Running 0 118m x.x.0.17 k8s-servicemesh-1 <none> <none>
meshb-service-entry.yaml內容
- apiVersion: networking.istio.io/v1alpha3
- kind: ServiceEntry
- metadata:
- name: meshb
- spec:
- endpoints:
- - address: x.x.0.17
- hosts:
- - AppMeshClient.mesh
- location: MESH_INTERNAL
- ports:
- - name: grpc
- number: 50000
- protocol: grpc
- resolution: STATIC
執(zhí)行如下命令ServiceEntry生效
- kubectl apply -f meshb-service-entry.yaml
- serviceentry.networking.istio.io/meshb created
再訪問頁面發(fā)現已經正常
備注:至此服務消費方在網格中向服務提供方發(fā)起調用。
五、日志校驗追蹤
查看服務消費方mesha日志
- kubectl logs -f mesha-b559fc4f4-m9752 -n default
- I'm from server mesh demo!
備注:服務消費方mesha從服務提供方meshb返回的信息。
查看服務提供方meshb日志
- # kubectl logs -f meshb-565945d794-wcb8z -n default
- 2021-11-05 09:21:19.828 INFO 1 --- [ main] o.s.b.w.embedded.tomcat.TomcatWebServer : Tomcat started on port(s): 8181 (http) with context path ''
- 2021-11-05 09:21:19.847 INFO 1 --- [ main] com.melon.test.MeshbApplication : Started MeshbApplication in 3.859 seconds (JVM running for 4.344)
- receiver client message: mesh demo!
備注:服務提供方meshb收到了服務消費方mesha的請求。
查看服務消費方mesha的envoy日志
- kubectl logs -f -l app=mesha -c istio-proxy -n default
- 2021-11-05T10:25:17.772317Z info xdsproxy connected to upstream XDS server: istiod.istio-system.svc:15012
- 2021-11-05T10:52:35.856445Z warning envoy config StreamAggregatedResources gRPC config stream closed: 14, transport is closing
- 2021-11-05T10:52:36.093048Z info xdsproxy connected to upstream XDS server: istiod.istio-system.svc:15012
- [2021-11-05T11:06:23.510Z] "GET /demo HTTP/1.1" 500 - via_upstream - "-" 0 287 37 36 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/95.0.4638.54 Safari/537.36" "49cd74d8-86eb-4ef5-bd71-8236d188c2f9" "10.69.31.156:30514" "10.166.1.117:7171" inbound|7171|| 127.0.0.6:41007 10.166.1.117:7171 10.166.0.0:20826 - default
- 2021-11-05T11:22:58.633956Z warning envoy config StreamAggregatedResources gRPC config stream closed: 14, transport is closing
- 2021-11-05T11:22:59.100387Z info xdsproxy connected to upstream XDS server: istiod.istio-system.svc:15012
- [2021-11-05T11:27:23.842Z] "POST /meshgrpc.Mesher/SayHello HTTP/2" 200 - via_upstream - "-" 17 33 371 319 "-" "grpc-java-netty/1.28.0" "79f2edbd-9c31-4265-87fb-38a594d9383b" "AppMeshClient.mesh:50000" "10.166.0.17:50000" outbound|50000||AppMeshClient.mesh 10.166.1.117:51914 240.240.0.63:50000 10.166.1.117:40544 - default
- [2021-11-05T11:27:23.527Z] "GET /demo HTTP/1.1" 200 - via_upstream - "-" 0 26 729 728 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/95.0.4638.54 Safari/537.36" "859344b6-012d-4457-a391-2b4d13aa3e35" "10.69.31.156:30514" "10.166.1.117:7171" inbound|7171|| 127.0.0.6:47065 10.166.1.117:7171 10.166.0.0:2410 - default
查看服務消費方meshb的envoy日志
- # kubectl logs -f -l app=meshb -c istio-proxy -n default
- 2021-11-05T09:57:12.490428Z warning envoy config StreamAggregatedResources gRPC config stream closed: 14, transport is closing
- 2021-11-05T09:57:12.765887Z info xdsproxy connected to upstream XDS server: istiod.istio-system.svc:15012
- 2021-11-05T10:24:48.767511Z warning envoy config StreamAggregatedResources gRPC config stream closed: 14, transport is closing
- 2021-11-05T10:24:48.914475Z info xdsproxy connected to upstream XDS server: istiod.istio-system.svc:15012
- 2021-11-05T10:57:52.604281Z warning envoy config StreamAggregatedResources gRPC config stream closed: 14, transport is closing
- 2021-11-05T10:57:52.757736Z info xdsproxy connected to upstream XDS server: istiod.istio-system.svc:15012
- 2021-11-05T11:26:56.551824Z warning envoy config StreamAggregatedResources gRPC config stream closed: 14, transport is closing
- 2021-11-05T11:26:56.987345Z info xdsproxy connected to upstream XDS server: istiod.istio-system.svc:15012
- [2021-11-05T11:27:23.844Z] "POST /meshgrpc.Mesher/SayHello HTTP/2" 200 - via_upstream - "-" 17 33 333 316 "-" "grpc-java-netty/1.28.0" "79f2edbd-9c31-4265-87fb-38a594d9383b" "AppMeshClient.mesh:50000" "10.166.0.17:50000" inbound|50000|| 127.0.0.6:44221 10.166.0.17:50000 10.166.1.117:51914 - default
登陸mesha的Pod中驗證
- kubectl describe pod/mesha-b559fc4f4-m9752 -n default
- [12:04:44root@mesha-b559fc4f4-m9752 /] C:2
- # curl -v AppMeshClient.mesh
- * About to connect() to AppMeshClient.mesh port 80 (#0)
- * Trying 240.240.0.63...
- * Connected to AppMeshClient.mesh (240.240.0.63) port 80 (#0)
- > GET / HTTP/1.1
- > User-Agent: curl/7.29.0
- > Host: AppMeshClient.mesh
- > Accept: */*
- >
- < HTTP/1.1 503 Service Unavailable
- < content-length: 91
- < content-type: text/plain
- < date: Fri, 05 Nov 2021 12:04:59 GMT
- < server: envoy
- <
- * Connection #0 to host AppMeshClient.mesh left intact
- upstream connect error or disconnect/reset before headers. reset reason: connection failure
備注:登陸mesha的Pod,通過訪問域名AppMeshClient.mesh,發(fā)現其自動分配IP「240.240.0.63」并指向sidecar「envoy」,即會把業(yè)務容器的流量通過DNS Proxy重定向到sidecar。
說明:通過查看服務消費方日志、服務提供方日志以及其數據面enovy日志,說明其調用在istio網格中進行。
本文轉載自微信公眾號「瓜農老梁」,可以通過以下二維碼關注。轉載本文請聯系瓜農老梁公眾號。