filebeat 6.3.0 host field 和 logstash 衝突

最近 filebeat 從 6.1.x 升級到 6.4.0，然後在 Dev 測試的時候就爆了，出現錯誤訊息：

object mapping for [host] tried to parse field [host] as object, but found a concrete value

 

第一個動作先翻 6.3.0 的 release note，果然加了 host.name field … 並且可能和 logstash 衝突 …

Add host.name field to all events, to avoid mapping conflicts. This could be breaking Logstash configs if you rely on the host field being a string

 

在論壇上也很多人在詢問，其中一個有官方回應「Problem with transfer Filebeat 6.1.3 > Logstash 6.1.3 > Elasticsearch 6.1.3」

It looks like your index was created with a new version of Beats that is implementing the ECS (Elastic Common Schema), but that the Logstash output isn’t quite in the same shape :weary:

ECS has a host.name field, which corresponds to the host name being output from Logstash. By adding a Mutate filter with a rename directive, you can move the host field to host.name with the field-reference syntax as below to align with the schema that you already have in Elasticsearch:

 

看起來 elastic 在重構自己的 Schema，所以在 6 之後的版本更新很快，然後問題也頗多 … 要解決這個問題的話就要處理 host 這個 field。

 

由於我的環境 filebeat 的版本有些還停在 6.1.x，但有些可以直上 6.3.x 之後的版本，所以必須要用 filter 做一些判斷

filter {
  if [beat][version] < "6.3.0" {
    mutate {
      rename => { "host" => "host.name" }
    }
  }
}

 

重啟生效後就搞定了 …