鍍金池/ 問答/Java  網(wǎng)絡(luò)安全/ elasticsearch+logstash+jdbc

elasticsearch+logstash+jdbc

logstash配置:

output {
   elasticsearch {
      hosts => ["192.168.8.8:9200"]
      index => "jdbc_index"
      document_id => "%{id}"
   }

   stdout {
      codec => json_lines
   }
}

上面配置兩種方式輸出的,為什么使用kibana顯示ES的記錄只有一條呢

clipboard.png
stdout輸出的就是正常的:

clipboard.png

搞了好久了,有前輩知道原因的嗎?

配置文件代碼:

input {
   stdin {
   }
   jdbc {
      jdbc_connection_string => "jdbc:mysql://192.168.12.222:3306/main"
      jdbc_user => "admin"
      jdbc_password => "123456"
      jdbc_driver_library => "/usr/local/logstash/mysql-connector-java-5.1.3.jar"
      jdbc_driver_class => "com.mysql.jdbc.Driver"
      jdbc_paging_enabled => "true"
      jdbc_page_size => "50000"
      statement_filepath => "/usr/local/logstash/config/jdbc.sql"
      #statement => "select inquiryId, inquiryNo, carTypeId, orgId, `status`, createTime, publishTime, failTime from qp_inquiry limit 500"
      schedule => "* * * * *"
      type => "jdbc_type"
      lowercase_column_names => "false"
   }
}

filter {
   json {
      source => "message"
      remove_field => ["message"]
   }
}

output {
   elasticsearch {
      hosts => ["192.168.8.8:9200"]
      index => "jdbc_index"
      document_id => "%{id}"
   }

   stdout {
      codec => json_lines
   }
}

sql語句:

SELECT
*
FROM
 (
  SELECT
   inquiryId,
   inquiryNo,
   carTypeId,
   orgId,
   `status`,
   createTime,
   publishTime,
   failTime
  FROM
   qp_inquiry
  LIMIT 50
 ) as `t1`
LIMIT 50000 OFFSET 0
回答
編輯回答
爆扎

看你的原始字段是沒有 id 這個字段的,你應(yīng)該指定一個存在的字段

2017年1月17日 17:03