eureka dns cluster

发表于 | 更新于 | 分类于 ， | 评论数：
本文字数： 16k | 阅读时长 ≈ 0:14

Netflix Eureka集群配置

Spring Cloud Netflix Eureka集群配置方式主要用两种：Static servers list config和dns，本文主要介绍基于DNS的方式搭建Eureka集群。

Static servers list config

client启动后从config中获取region和zone以及serviceUrl，进行服务注册、发现，这种静态集群模式无法动态扩容，一旦新增节点，只能挨个修改server和client端的配置文件，大致配置如下：

1
2
3
4
5
6
7
8
9
10
11
eureka:
  server:
    enable-self-preservation: false
  instance:
    prefer-ip-address: true
    instance-id: ${spring.cloud.client.ip-address}:${spring.application.name}:${server.port}
  client:
    register-with-eureka: true
    fetch-registry: true
    service-url:
      defaultZone: http://10.25.102.174:8082/eureka/,http://10.25.102.174:8083/eureka/

DNS configuration

基于DNS的Eureka Cluster架构图如下：

eureka-dns-cluster-architecture-diagram

eureka-client-invoke-process

简单来说就是使用DNS服务器管理Eureka Server的地址，弹性伸缩对客户端没有影响。

大致流程如下：

  1. 客户端开启DNS方式获取serviceUrl：use-dns-for-fetching-service-urls: true；
  2. 根据client.region配置的区域region，查询DNS区域配置文件中该区域下的所有可用区zone；
  3. 循环所有可用区zone，获取可用区配置的eureka-server地址；
  4. 拼接成完整的serviceUrl并加入serviceUrls列表中，serviceUrl格式为："http://" + ec2Url + ":" + clientConfig.getEurekaServerPort() + "/" + clientConfig.getEurekaServerURLContext() + "/";。

只要解决DNS服务器配置就可以达到动态集群的效果，关于如何搭建DNS服务器集群，请移步另一篇文章：CentOS 搭建DNS主从服务器集群，文中详细介绍了如何在区域配置文件中配置TXT记录，本文不在赘述。

1.DNS 区域文件配置TXT记录

zts.local.zone区域配置文件中有关eureka的配置：

1
2
3
4
5
6
7
8
9
10
; eureka cluster配置
txt.shanghai IN TXT "defaultZone.zts.local"
txt.defaultZone IN TXT "10.29.181.56" "10.29.181.57" "10.29.181.58" "10.29.181.60"
;通过查看Eureka的源码，它的实现是通过寻找txt.beijing-a.zts.local（txt.region.eureka-server-d-n-s-name）获取"beijing-a.zts.local" "beijing-b.zts.local"的zone：beijing-a和beijing-b，
;然后再去获取txt.beijing-a.zts.local和txt.beijing-b.zts.local中对应的ServiceUrls的数据，也就是IP地址: "10.29.181.61" "10.29.181.62"
;这样服务间就可以获取到固定端口下的不同IP的机器的注册中心服务地址，并相互注册
txt.beijing IN TXT "beijing-a.zts.local" "beijing-b.zts.local"
txt.beijing-a IN TXT "10.29.181.61" "10.29.181.62"
txt.beijing-b IN TXT "10.29.181.63" "10.29.181.64"

2.Eureka Server/Client端配置

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
---
spring:
  application:
    name: eureka-server
server:
  port: 8081
eureka:
  server:
    enable-self-preservation: false
  instance:
    prefer-ip-address: true
    instance-id: ${spring.cloud.client.ip-address}:${spring.application.name}:${server.port}
  client:
    ## 是否将自己注册到eureka
    register-with-eureka: true
    ## 是否获取注册信息到本地
    fetch-registry: true
    ## 开启DNS方式获取serviceUrl,默认为false
    use-dns-for-fetching-service-urls: true
    ## DNS域名,获取其他信息将以该域名为根域名
    eureka-server-d-n-s-name: zts.local
    ## 当前应用所在区域,默认为us-east-1
    region: beijing
    ## 配置中心的eureka目录
    eureka-server-u-r-l-context: eureka
    ## 映射其他eurekaServer的端口，这里注意采用这种方式 server.port和这个端口最好一致
    ## 因为dns是控制地址的变化，端口不变，所以不像前面的哪些配置方式可以自己定义url和port。
    ## 所以每个IP都是用的相同的port进行注册中心服务的部署
    eureka-server-port: 8081
    ## 获取serviceUrl时候是否优先获取相同zone的列表(如果获取为空则获取所在region第一个zone),如果为false则优先获取不在相同zone的列表
    prefer-same-zone-eureka: true

3.测试

修改客户端DNS服务器配置为10.29.181.61，DNS服务器的域名为zts.local

依次启动4台Eureka Server服务器，查看启动日志（删减版的日志），：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
2018-11-20 16:17:14.962  INFO 49750 --- [           main] c.n.d.s.r.aws.ConfigClusterResolver      : Resolving eureka endpoints via DNS: txt.beijing.zts.local (<------注意这里)
2018-11-20 16:17:15.012  INFO 49750 --- [           main] com.netflix.discovery.DiscoveryClient    : Disable delta property : false
2018-11-20 16:17:15.262  INFO 49750 --- [           main] com.netflix.discovery.DiscoveryClient    : Discovery Client initialized at timestamp 1542701835260 with initial instances count: 5
2018-11-20 16:17:15.326  INFO 49750 --- [           main] c.n.eureka.DefaultEurekaServerContext    : Initializing ...
2018-11-20 16:17:15.559  INFO 49750 --- [           main] c.n.eureka.cluster.PeerEurekaNodes       : Replica node URL:  http://10.29.181.62:8081/eureka/
2018-11-20 16:17:15.560  INFO 49750 --- [           main] c.n.eureka.cluster.PeerEurekaNodes       : Replica node URL:  http://10.29.181.64:8081/eureka/
2018-11-20 16:17:15.560  INFO 49750 --- [           main] c.n.eureka.cluster.PeerEurekaNodes       : Replica node URL:  http://10.29.181.61:8081/eureka/
2018-11-20 16:17:16.052  INFO 49750 --- [      Thread-14] o.s.c.n.e.server.EurekaServerBootstrap   : isAws returned false
2018-11-20 16:17:16.053  INFO 49750 --- [      Thread-14] o.s.c.n.e.server.EurekaServerBootstrap   : Initialized server context
2018-11-20 16:17:16.056  INFO 49750 --- [           main] c.z.e.server.EurekaServerApplication     : Started EurekaServerApplication in 7.112 seconds (JVM running for 7.625)
2018-11-20 16:17:16.060  INFO 49750 --- [      Thread-14] c.n.e.registry.AbstractInstanceRegistry  : Registered instance EUREKA-SERVER/10.29.181.61:eureka-server:8081 with status UP (replication=true)
2018-11-20 16:17:16.061  INFO 49750 --- [      Thread-14] c.n.e.registry.AbstractInstanceRegistry  : Registered instance EUREKA-SERVER/10.29.181.62:eureka-server:8081 with status UP (replication=true)
2018-11-20 16:17:16.061  INFO 49750 --- [      Thread-14] c.n.e.registry.AbstractInstanceRegistry  : Registered instance EUREKA-SERVER/10.29.181.64:eureka-server:8081 with status UP (replication=true)
2018-11-20 16:17:16.062  INFO 49750 --- [      Thread-14] c.n.e.registry.AbstractInstanceRegistry  : Registered instance EUREKA-SERVER/10.29.181.63:eureka-server:8081 with status UP (replication=true)
2018-11-20 16:17:16.062  INFO 49750 --- [      Thread-14] c.n.e.registry.AbstractInstanceRegistry  : Registered instance ACCOUNT-CENTER/10.25.102.174:account-center:8891 with status UP (replication=true)
2018-11-20 16:17:16.062  INFO 49750 --- [      Thread-14] c.n.e.r.PeerAwareInstanceRegistryImpl    : Got 5 instances from neighboring DS node
2018-11-20 16:17:16.062  INFO 49750 --- [      Thread-14] c.n.e.r.PeerAwareInstanceRegistryImpl    : Renew threshold is: 8
2018-11-20 16:17:16.062  INFO 49750 --- [      Thread-14] c.n.e.r.PeerAwareInstanceRegistryImpl    : Changing status to UP
2018-11-20 16:17:16.070  INFO 49750 --- [nfoReplicator-0] com.netflix.discovery.DiscoveryClient    : DiscoveryClient_EUREKA-SERVER/10.29.181.63:eureka-server:8081 - registration status: 204
2018-11-20 16:17:16.070  INFO 49750 --- [      Thread-14] e.s.EurekaServerInitializerConfiguration : Started Eureka Server
2018-11-20 16:17:16.310  INFO 49750 --- [nio-8081-exec-1] o.a.c.c.C.[Tomcat].[localhost].[/]       : Initializing Spring FrameworkServlet 'dispatcherServlet'
2018-11-20 16:17:16.310  INFO 49750 --- [nio-8081-exec-1] o.s.web.servlet.DispatcherServlet        : FrameworkServlet 'dispatcherServlet': initialization started
2018-11-20 16:17:16.327  INFO 49750 --- [nio-8081-exec-1] o.s.web.servlet.DispatcherServlet        : FrameworkServlet 'dispatcherServlet': initialization completed in 17 ms
2018-11-20 16:17:16.578  INFO 49750 --- [nio-8081-exec-1] c.n.e.registry.AbstractInstanceRegistry  : Registered instance EUREKA-SERVER/10.29.181.63:eureka-server:8081 with status UP (replication=true)
2018-11-20 16:17:17.457  INFO 49750 --- [nio-8081-exec-2] c.n.e.registry.AbstractInstanceRegistry  : Registered instance EUREKA-SERVER/10.29.181.63:eureka-server:8081 with status UP (replication=true)

Eureka Client端

Account服务配置

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
logging:
  level:
    org.springframework.boot: info
    org.springframework.cloud: debug
spring:
  application:
    name: account-center
server:
  port: 8891
eureka:
  server:
    enable-self-preservation: false
  instance:
    prefer-ip-address: true
    instance-id: ${spring.cloud.client.ip-address}:${spring.application.name}:${server.port}
    metadata-map:
      configPath: ${server.servlet.context-path}
    home-page-url-path: ${server.servlet.context-path}
    ## 服务过期时间配置,超过这个时间没有接收到心跳EurekaServer就会将这个实例剔除
    lease-expiration-duration-in-seconds: 10
    ## 服务刷新时间配置，每隔这个时间会主动心跳一次
    lease-renewal-interval-in-seconds: 5
  client:
    ## 是否将自己注册到eureka
    register-with-eureka: true
    ## 是否获取注册信息到本地
    fetch-registry: true
    ## 开启DNS方式获取serviceUrl,默认为false
    use-dns-for-fetching-service-urls: true
    ## DNS域名,获取其他信息将以该域名为根域名
    eureka-server-d-n-s-name: zts.local
    ## 当前应用所在区域,默认为us-east-1
    region: beijing
    ## 配置中心的eureka目录
    eureka-server-u-r-l-context: eureka
    ## 映射其他eurekaServer的端口，这里注意采用这种方式 server.port和这个端口最好一致
    ## 因为dns是控制地址的变化，端口不变，所以不像前面的哪些配置方式可以自己定义url和port。
    ## 所以每个IP都是用的相同的port进行注册中心服务的部署
    eureka-server-port: 8081
    ## 获取serviceUrl时候是否优先获取相同zone的列表(如果获取为空则获取所在region第一个zone),如果为false则优先获取不在相同zone的列表
    prefer-same-zone-eureka: true

Account启动日志：

1
2
3
4
5
6
2018-11-20 16:11:05.373  INFO 5824 --- [  restartedMain] c.n.d.s.r.aws.ConfigClusterResolver      : Resolving eureka endpoints via DNS: txt.beijing.zts.local (<------看这里)
2018-11-20 16:11:05.529  INFO 5824 --- [  restartedMain] com.netflix.discovery.DiscoveryClient    : Disable delta property : false
2018-11-20 16:11:05.529  INFO 5824 --- [  restartedMain] com.netflix.discovery.DiscoveryClient    : Single vip registry refresh property : null
2018-11-20 16:11:05.529  INFO 5824 --- [  restartedMain] com.netflix.discovery.DiscoveryClient    : Force full registry fetch : false
2018-11-20 16:11:05.529  INFO 5824 --- [  restartedMain] com.netflix.discovery.DiscoveryClient    : Application is null : false
201

打开Eureka console，查看注册效果：
查看效果

参考资料

1.com.netflix.discovery.endpoint.EndpointUtils.getServiceUrlsFromDNS

版本：eureka-client:1.9.3 

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
/**
    * Get the list of all eureka service urls for the eureka client to talk to.
    *
    * @param clientConfig the clientConfig to use
    * @param zone the zone in which the client resides
    * @param randomizer a randomizer to randomized returned urls, if loading from dns
    *
    * @return The list of all eureka service urls for the eureka client to talk to.
    */
   public static List<String> getDiscoveryServiceUrls(EurekaClientConfig clientConfig, String zone, ServiceUrlRandomizer randomizer) {
       boolean shouldUseDns = clientConfig.shouldUseDnsForFetchingServiceUrls();
       if (shouldUseDns) {
           return getServiceUrlsFromDNS(clientConfig, zone, clientConfig.shouldPreferSameZoneEureka(), randomizer);
       }
       return getServiceUrlsFromConfig(clientConfig, zone, clientConfig.shouldPreferSameZoneEureka());
   }
   /**
    * Get the list of all eureka service urls from DNS for the eureka client to
    * talk to. The client picks up the service url from its zone and then fails over to
    * other zones randomly. If there are multiple servers in the same zone, the client once
    * again picks one randomly. This way the traffic will be distributed in the case of failures.
    *
    * @param clientConfig the clientConfig to use
    * @param instanceZone The zone in which the client resides.
    * @param preferSameZone true if we have to prefer the same zone as the client, false otherwise.
    * @param randomizer a randomizer to randomized returned urls
    *
    * @return The list of all eureka service urls for the eureka client to talk to.
    */
   public static List<String> getServiceUrlsFromDNS(EurekaClientConfig clientConfig, String instanceZone, boolean preferSameZone, ServiceUrlRandomizer randomizer) {
       String region = getRegion(clientConfig);
       // Get zone-specific DNS names for the given region so that we can get a
       // list of available zones
       Map<String, List<String>> zoneDnsNamesMap = getZoneBasedDiscoveryUrlsFromRegion(clientConfig, region);
       Set<String> availableZones = zoneDnsNamesMap.keySet();
       List<String> zones = new ArrayList<String>(availableZones);
       if (zones.isEmpty()) {
           throw new RuntimeException("No available zones configured for the instanceZone " + instanceZone);
       }
       int zoneIndex = 0;
       boolean zoneFound = false;
       for (String zone : zones) {
           logger.debug("Checking if the instance zone {} is the same as the zone from DNS {}", instanceZone, zone);
           if (preferSameZone) {
               if (instanceZone.equalsIgnoreCase(zone)) {
                   zoneFound = true;
               }
           } else {
               if (!instanceZone.equalsIgnoreCase(zone)) {
                   zoneFound = true;
               }
           }
           if (zoneFound) {
               logger.debug("The zone index from the list {} that matches the instance zone {} is {}",
                       zones, instanceZone, zoneIndex);
               break;
           }
           zoneIndex++;
       }
       if (zoneIndex >= zones.size()) {
           if (logger.isWarnEnabled()) {
               logger.warn("No match for the zone {} in the list of available zones {}",
                       instanceZone, zones.toArray());
           }
       } else {
           // Rearrange the zones with the instance zone first
           for (int i = 0; i < zoneIndex; i++) {
               String zone = zones.remove(0);
               zones.add(zone);
           }
       }
       // Now get the eureka urls for all the zones in the order and return it
       List<String> serviceUrls = new ArrayList<String>();
       for (String zone : zones) {
           for (String zoneCname : zoneDnsNamesMap.get(zone)) {
               List<String> ec2Urls = new ArrayList<String>(getEC2DiscoveryUrlsFromZone(zoneCname, DiscoveryUrlType.CNAME));
               // Rearrange the list to distribute the load in case of multiple servers
               if (ec2Urls.size() > 1) {
                   randomizer.randomize(ec2Urls);
               }
               for (String ec2Url : ec2Urls) {
                   StringBuilder sb = new StringBuilder()
                           .append("http://")
                           .append(ec2Url)
                           .append(":")
                           .append(clientConfig.getEurekaServerPort());
                   if (clientConfig.getEurekaServerURLContext() != null) {
                       if (!clientConfig.getEurekaServerURLContext().startsWith("/")) {
                           sb.append("/");
                       }
                       sb.append(clientConfig.getEurekaServerURLContext());
                       if (!clientConfig.getEurekaServerURLContext().endsWith("/")) {
                           sb.append("/");
                       }
                   } else {
                       sb.append("/");
                   }
                   String serviceUrl = sb.toString();
                   logger.debug("The EC2 url is {}", serviceUrl);
                   serviceUrls.add(serviceUrl);
               }
           }
       }
       // Rearrange the fail over server list to distribute the load
       String primaryServiceUrl = serviceUrls.remove(0);
       randomizer.randomize(serviceUrls);
       serviceUrls.add(0, primaryServiceUrl);
       if (logger.isDebugEnabled()) {
           logger.debug("This client will talk to the following serviceUrls in order : {} ",
                   (Object) serviceUrls.toArray());
       }
       return serviceUrls;
   }

2. dns config配置参考文章