org.apache.flume.sink.hbase
Class RegexHbaseEventSerializer

java.lang.Object
  extended by org.apache.flume.sink.hbase.RegexHbaseEventSerializer
All Implemented Interfaces:
Configurable, ConfigurableComponent, HbaseEventSerializer

public class RegexHbaseEventSerializer
extends Object
implements HbaseEventSerializer

An HbaseEventSerializer which parses columns based on a supplied regular expression and column name list. Note that if the regular expression does not return the correct number of groups for a particular event, or it does not correctly match an event, the event is silently dropped. Row keys for each event consist of a timestamp concatenated with an identifier which enforces uniqueness of keys across flume agents. See static constant variables for configuration options.


Field Summary
protected  byte[] cf
           
static String CHARSET_CONFIG
          What charset to use when serializing into HBase's byte arrays
static String CHARSET_DEFAULT
           
static String COL_NAME_CONFIG
          Comma separated list of column names to place matching groups in.
static String COLUMN_NAME_DEFAULT
           
static String DEPOSIT_HEADERS_CONFIG
          Whether to deposit event headers into corresponding column qualifiers
static boolean DEPOSIT_HEADERS_DEFAULT
           
static String IGNORE_CASE_CONFIG
          Whether to ignore case when performing regex matches.
static boolean INGORE_CASE_DEFAULT
           
protected static AtomicInteger nonce
           
protected static String randomKey
           
static String REGEX_CONFIG
          Regular expression used to parse groups from event data.
static String REGEX_DEFAULT
           
static String ROW_KEY_INDEX_CONFIG
          Index of the row key in matched regex groups
static String ROW_KEY_NAME
          Placeholder in colNames for row key
 
Constructor Summary
RegexHbaseEventSerializer()
           
 
Method Summary
 void close()
           
 void configure(ComponentConfiguration conf)
           
 void configure(Context context)
           Request the implementing class to (re)configure itself.
 List<org.apache.hadoop.hbase.client.Row> getActions()
          Get the actions that should be written out to hbase as a result of this event.
 List<org.apache.hadoop.hbase.client.Increment> getIncrements()
           
protected  byte[] getRowKey()
           
protected  byte[] getRowKey(Calendar cal)
          Returns a row-key with the following format: [time in millis]-[random key]-[nonce]
 void initialize(Event event, byte[] columnFamily)
          Initialize the event serializer.
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Field Detail

REGEX_CONFIG

public static final String REGEX_CONFIG
Regular expression used to parse groups from event data.

See Also:
Constant Field Values

REGEX_DEFAULT

public static final String REGEX_DEFAULT
See Also:
Constant Field Values

IGNORE_CASE_CONFIG

public static final String IGNORE_CASE_CONFIG
Whether to ignore case when performing regex matches.

See Also:
Constant Field Values

INGORE_CASE_DEFAULT

public static final boolean INGORE_CASE_DEFAULT
See Also:
Constant Field Values

COL_NAME_CONFIG

public static final String COL_NAME_CONFIG
Comma separated list of column names to place matching groups in.

See Also:
Constant Field Values

COLUMN_NAME_DEFAULT

public static final String COLUMN_NAME_DEFAULT
See Also:
Constant Field Values

ROW_KEY_INDEX_CONFIG

public static final String ROW_KEY_INDEX_CONFIG
Index of the row key in matched regex groups

See Also:
Constant Field Values

ROW_KEY_NAME

public static final String ROW_KEY_NAME
Placeholder in colNames for row key

See Also:
Constant Field Values

DEPOSIT_HEADERS_CONFIG

public static final String DEPOSIT_HEADERS_CONFIG
Whether to deposit event headers into corresponding column qualifiers

See Also:
Constant Field Values

DEPOSIT_HEADERS_DEFAULT

public static final boolean DEPOSIT_HEADERS_DEFAULT
See Also:
Constant Field Values

CHARSET_CONFIG

public static final String CHARSET_CONFIG
What charset to use when serializing into HBase's byte arrays

See Also:
Constant Field Values

CHARSET_DEFAULT

public static final String CHARSET_DEFAULT
See Also:
Constant Field Values

nonce

protected static final AtomicInteger nonce

randomKey

protected static String randomKey

cf

protected byte[] cf
Constructor Detail

RegexHbaseEventSerializer

public RegexHbaseEventSerializer()
Method Detail

configure

public void configure(Context context)
Description copied from interface: Configurable

Request the implementing class to (re)configure itself.

When configuration parameters are changed, they must be reflected by the component asap.

There are no thread safety guarrantees on when configure might be called.

Specified by:
configure in interface Configurable

configure

public void configure(ComponentConfiguration conf)
Specified by:
configure in interface ConfigurableComponent

initialize

public void initialize(Event event,
                       byte[] columnFamily)
Description copied from interface: HbaseEventSerializer
Initialize the event serializer.

Specified by:
initialize in interface HbaseEventSerializer

getRowKey

protected byte[] getRowKey(Calendar cal)
Returns a row-key with the following format: [time in millis]-[random key]-[nonce]


getRowKey

protected byte[] getRowKey()

getActions

public List<org.apache.hadoop.hbase.client.Row> getActions()
                                                    throws FlumeException
Description copied from interface: HbaseEventSerializer
Get the actions that should be written out to hbase as a result of this event. This list is written to hbase using the HBase batch API.

Specified by:
getActions in interface HbaseEventSerializer
Returns:
List of Row which are written as such to HBase. 0.92 increments do not implement Row, so this is not generic.
Throws:
FlumeException

getIncrements

public List<org.apache.hadoop.hbase.client.Increment> getIncrements()
Specified by:
getIncrements in interface HbaseEventSerializer

close

public void close()
Specified by:
close in interface HbaseEventSerializer


Copyright © 2009-2015 Apache Software Foundation. All Rights Reserved.