Elasticsearch 問題分析：No data nodes with HTTP-enabled available

標籤：set   private   ipaddress   esc   parse   message   tin   null   邏輯   

環境：ES-5.4.0版本，部署方式：3master+2client+3datanode

說明：datanode和client都配置了http.enabled: false，程式在寫資料時報錯：No data nodes with HTTP-enabled available

源碼分析：

public static void filterNonDataNodesIfNeeded(Settings settings, Log log) {    if (!settings.getNodesDataOnly()) {        return;    }    RestClient bootstrap = new RestClient(settings);    try  {        String message = "No data nodes with HTTP-enabled available";        List<NodeInfo> dataNodes = bootstrap.getHttpDataNodes();
　　　　// 找不到dataNodes就會報錯        if (dataNodes.isEmpty()) {            throw new EsHadoopIllegalArgumentException(message);        }        ...    } finally {        bootstrap.close();    }}

接下來看看RestClient.getHttpDataNodes()方法的取值邏輯

public List<NodeInfo> getHttpDataNodes() {    List<NodeInfo> nodes = getHttpNodes(false);
　　// 遍曆上面擷取到的節點    Iterator<NodeInfo> it = nodes.iterator();    while (it.hasNext()) {        NodeInfo node = it.next();
　　　　// 如果不是資料節點，則移除        if (!node.isData()) {            it.remove();        }    }    return nodes;}
// 擷取http節點_nodes/httppublic List<NodeInfo> getHttpNodes(boolean clientNodeOnly) {
　　// 通過es介面“_nodes/http”來擷取nodes的資訊    Map<String, Map<String, Object>> nodesData = get("_nodes/http", "nodes");    List<NodeInfo> nodes = new ArrayList<NodeInfo>();    for (Entry<String, Map<String, Object>> entry : nodesData.entrySet()) {        NodeInfo node = new NodeInfo(entry.getKey(), entry.getValue());
　　　　// 如果不是尋找client節點，則只要節點運行網路訪問就可以add了；如果尋找client節點，則還要通過isClient驗證才能add        if (node.hasHttp() && (!clientNodeOnly || node.isClient())) {            nodes.add(node);        }    }    return nodes;}

 最後再來看看node.hasHttp()，isClient()，isData()的方法

    private final String id;    private final String name;    private final String host;    private final String ip;    private final String publishAddress;    private final boolean hasHttp;    private final boolean isClient;    private final boolean isData;    private final boolean isIngest;    public NodeInfo(String id, Map<String, Object> map) {        this.id = id;        EsMajorVersion version = EsMajorVersion.parse((String) map.get("version"));        this.name = (String) map.get("name");        this.host = (String) map.get("host");        this.ip = (String) map.get("ip");
　　　　// 5.0以下版本的分支        if (version.before(EsMajorVersion.V_5_X)) {            Map<String, Object> attributes = (Map<String, Object>) map.get("attributes");            if (attributes == null) {                this.isClient = false;                this.isData = true;            } else {                String data = (String) attributes.get("data");                this.isClient = data == null ? true : !Boolean.parseBoolean(data);                this.isData = data == null ? true : Boolean.parseBoolean(data);            }            this.isIngest = false;
　　　　// 5.0版本以上的分支        } else {            List<String> roles = (List<String>) map.get("roles");
　　　　　　// 如果roles列表中不包含"data"，則此節點是client            this.isClient = roles.contains("data") == false;
　　　　　　// 如果roles列表中包含"data"，則此節點是data            this.isData = roles.contains("data");
　　　　　　// 如果roles列表中包含"ingest"，則此節點是ingest            this.isIngest = roles.contains("ingest");        }        Map<String, Object> httpMap = (Map<String, Object>) map.get("http");
　　　　// 如果節點資料中包含key：http        if (httpMap != null) {            String addr = (String) httpMap.get("publish_address");
　　　　　　// 如果http資料中包含key：publish_address            if (addr != null) {                StringUtils.IpAndPort ipAndPort = StringUtils.parseIpAddress(addr);                this.publishAddress = ipAndPort.ip + ":" + ipAndPort.port;
　　　　　　　　// 則此節點可以提供http服務，即：http.enabled: true                this.hasHttp = true;            } else {                this.publishAddress = null;                this.hasHttp = false;            }        } else {            this.publishAddress = null;            this.hasHttp = false;        }    }

 從上面的源碼分析可以得出：如果一個data節點不配置http.enabled：true，則此節點不會被getHttpDataNodes()方法搜尋到，那麼就會直接拋出異常：No data nodes with HTTP-enabled available

解決的方法無非兩種：

第一：資料節點配置 http.enabled：true

第二：繞過filterNonDataNodesIfNeeded()校正，需要settings.getNodesDataOnly()返回false；看下面源碼可知，預設es.nodes.data.only是true，在用戶端中將其設定為false即可。

/** Clients only */String ES_NODES_CLIENT_ONLY = "es.nodes.client.only";String ES_NODES_CLIENT_ONLY_DEFAULT = "false";/** Data only */String ES_NODES_DATA_ONLY = "es.nodes.data.only";String ES_NODES_DATA_ONLY_DEFAULT = "true";/** Ingest only */String ES_NODES_INGEST_ONLY = "es.nodes.ingest.only";String ES_NODES_INGEST_ONLY_DEFAULT = "false";/** WAN only */String ES_NODES_WAN_ONLY = "es.nodes.wan.only";String ES_NODES_WAN_ONLY_DEFAULT = "false";...public boolean getNodesDataOnly() {    // by default, if not set, return a value compatible with the other settings
　　// 預設es.nodes.data.only是true，在用戶端中將其設定為false即可    return Booleans.parseBoolean(getProperty(ES_NODES_DATA_ONLY), !getNodesWANOnly() && !getNodesClientOnly() && !getNodesIngestOnly());}public boolean getNodesIngestOnly() {    return Booleans.parseBoolean(getProperty(ES_NODES_INGEST_ONLY, ES_NODES_INGEST_ONLY_DEFAULT));}public boolean getNodesClientOnly() {    return Booleans.parseBoolean(getProperty(ES_NODES_CLIENT_ONLY, ES_NODES_CLIENT_ONLY_DEFAULT));}public boolean getNodesWANOnly() {    return Booleans.parseBoolean(getProperty(ES_NODES_WAN_ONLY, ES_NODES_WAN_ONLY_DEFAULT));}

 

Elasticsearch 問題分析：No data nodes with HTTP-enabled available