org.apache.flume.sink.elasticsearch

Class ElasticSearchLogStashEventSerializer

java.lang.Object

org.apache.flume.sink.elasticsearch.ElasticSearchLogStashEventSerializer

All Implemented Interfaces:

Configurable, ConfigurableComponent, ElasticSearchEventSerializer

public class ElasticSearchLogStashEventSerializer
extends Object
implements ElasticSearchEventSerializer

Serialize flume events into the same format LogStash uses

This can be used to send events to ElasticSearch and use clients such as Kabana which expect Logstash formated indexes

 {
    "@timestamp": "2010-12-21T21:48:33.309258Z",
    "@tags": [ "array", "of", "tags" ],
    "@type": "string",
    "@source": "source of the event, usually a URL."
    "@source_host": ""
    "@source_path": ""
    "@fields":{
       # a set of fields for this event
       "user": "jordan",
       "command": "shutdown -r":
     }
     "@message": "the original plain-text message"
   }
 

If the following headers are present, they will map to the above logstash output as long as the logstash fields are not already present.

  timestamp: long -> @timestamp:Date
  host: String -> @source_host: String
  src_path: String -> @source_path: String
  type: String -> @type: String
  source: String -> @source: String
 

See Also:

://github.com/logstash/logstash/wiki/logstash%27s-internal-message- format

Field Summary

Fields inherited from interface org.apache.flume.sink.elasticsearch.ElasticSearchEventSerializer

charset

Constructor Summary

Constructors

Constructor and Description
ElasticSearchLogStashEventSerializer()

Method Summary

Modifier and Type	Method and Description
void	configure(ComponentConfiguration conf)
void	configure(Context context) Request the implementing class to (re)configure itself.
org.elasticsearch.common.xcontent.XContentBuilder	getContentBuilder(Event event) Return an BytesStream made up of the serialized flume event

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

ElasticSearchLogStashEventSerializer

public ElasticSearchLogStashEventSerializer()

Method Detail

getContentBuilder

public org.elasticsearch.common.xcontent.XContentBuilder getContentBuilder(Event event)
                                                                    throws IOException

Description copied from interface: ElasticSearchEventSerializer

Return an BytesStream made up of the serialized flume event

Specified by:

getContentBuilder in interface ElasticSearchEventSerializer

Parameters:

event - The flume event to serialize

Returns:

A BytesStream used to write to ElasticSearch

Throws:

IOException - If an error occurs during serialization

configure

public void configure(Context context)

Description copied from interface: Configurable

Request the implementing class to (re)configure itself.

When configuration parameters are changed, they must be reflected by the component asap.

There are no thread safety guarantees on when configure might be called.

Specified by:

configure in interface Configurable

configure

public void configure(ComponentConfiguration conf)

Specified by:

configure in interface ConfigurableComponent