org.apache.flume.sink.hbase

Class RegexHbaseEventSerializer

java.lang.Object

org.apache.flume.sink.hbase.RegexHbaseEventSerializer

All Implemented Interfaces:

Configurable, ConfigurableComponent, HbaseEventSerializer

public class RegexHbaseEventSerializer
extends Object
implements HbaseEventSerializer

An HbaseEventSerializer which parses columns based on a supplied regular expression and column name list.

Note that if the regular expression does not return the correct number of groups for a particular event, or it does not correctly match an event, the event is silently dropped.

Row keys for each event consist of a timestamp concatenated with an identifier which enforces uniqueness of keys across flume agents.

See static constant variables for configuration options.

Field Summary

Fields

Modifier and Type	Field and Description
protected byte[]	cf
static String	CHARSET_CONFIG What charset to use when serializing into HBase's byte arrays
static String	CHARSET_DEFAULT
static String	COL_NAME_CONFIG Comma separated list of column names to place matching groups in.
static String	COLUMN_NAME_DEFAULT
static String	DEPOSIT_HEADERS_CONFIG Whether to deposit event headers into corresponding column qualifiers
static boolean	DEPOSIT_HEADERS_DEFAULT
static String	IGNORE_CASE_CONFIG Whether to ignore case when performing regex matches.
static boolean	INGORE_CASE_DEFAULT
protected static AtomicInteger	nonce
protected static String	randomKey
static String	REGEX_CONFIG Regular expression used to parse groups from event data.
static String	REGEX_DEFAULT
static String	ROW_KEY_INDEX_CONFIG Index of the row key in matched regex groups
static String	ROW_KEY_NAME Placeholder in colNames for row key

Constructor Summary

Constructors

Constructor and Description
RegexHbaseEventSerializer()

Method Summary

Methods

Modifier and Type	Method and Description
void	close()
void	configure(ComponentConfiguration conf)
void	configure(Context context) Request the implementing class to (re)configure itself.
List<org.apache.hadoop.hbase.client.Row>	getActions() Get the actions that should be written out to hbase as a result of this event.
List<org.apache.hadoop.hbase.client.Increment>	getIncrements()
protected byte[]	getRowKey()
protected byte[]	getRowKey(Calendar cal) Returns a row-key with the following format: [time in millis]-[random key]-[nonce]
void	initialize(Event event, byte[] columnFamily) Initialize the event serializer.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

REGEX_CONFIG

public static final String REGEX_CONFIG

Regular expression used to parse groups from event data.

See Also:

Constant Field Values

REGEX_DEFAULT

public static final String REGEX_DEFAULT

See Also:

Constant Field Values

IGNORE_CASE_CONFIG

public static final String IGNORE_CASE_CONFIG

Whether to ignore case when performing regex matches.

See Also:

Constant Field Values

INGORE_CASE_DEFAULT

public static final boolean INGORE_CASE_DEFAULT

See Also:

Constant Field Values

COL_NAME_CONFIG

public static final String COL_NAME_CONFIG

Comma separated list of column names to place matching groups in.

See Also:

Constant Field Values

COLUMN_NAME_DEFAULT

public static final String COLUMN_NAME_DEFAULT

See Also:

Constant Field Values

ROW_KEY_INDEX_CONFIG

public static final String ROW_KEY_INDEX_CONFIG

Index of the row key in matched regex groups

See Also:

Constant Field Values

ROW_KEY_NAME

public static final String ROW_KEY_NAME

Placeholder in colNames for row key

See Also:

Constant Field Values

DEPOSIT_HEADERS_CONFIG

public static final String DEPOSIT_HEADERS_CONFIG

Whether to deposit event headers into corresponding column qualifiers

See Also:

Constant Field Values

DEPOSIT_HEADERS_DEFAULT

public static final boolean DEPOSIT_HEADERS_DEFAULT

See Also:

Constant Field Values

CHARSET_CONFIG

public static final String CHARSET_CONFIG

What charset to use when serializing into HBase's byte arrays

See Also:

Constant Field Values

CHARSET_DEFAULT

public static final String CHARSET_DEFAULT

See Also:

Constant Field Values

nonce

protected static final AtomicInteger nonce

randomKey

protected static String randomKey

cf

protected byte[] cf

Constructor Detail

RegexHbaseEventSerializer

public RegexHbaseEventSerializer()

Method Detail

configure

public void configure(Context context)

Description copied from interface: Configurable

Request the implementing class to (re)configure itself.

When configuration parameters are changed, they must be reflected by the component asap.

There are no thread safety guarantees on when configure might be called.

Specified by:

configure in interface Configurable

configure

public void configure(ComponentConfiguration conf)

Specified by:

configure in interface ConfigurableComponent

initialize

public void initialize(Event event,
              byte[] columnFamily)

Description copied from interface: HbaseEventSerializer

Initialize the event serializer.

Specified by:

initialize in interface HbaseEventSerializer

Parameters:

event - Event to be written to HBase

columnFamily - Column family to write to

getRowKey

protected byte[] getRowKey(Calendar cal)

Returns a row-key with the following format: [time in millis]-[random key]-[nonce]

getRowKey

protected byte[] getRowKey()

getActions

public List<org.apache.hadoop.hbase.client.Row> getActions()
                                                    throws FlumeException

Description copied from interface: HbaseEventSerializer

Get the actions that should be written out to hbase as a result of this event. This list is written to hbase using the HBase batch API.

Specified by:

getActions in interface HbaseEventSerializer

Returns:

List of Row which are written as such to HBase. 0.92 increments do not implement Row, so this is not generic.

Throws:

FlumeException

getIncrements

public List<org.apache.hadoop.hbase.client.Increment> getIncrements()

Specified by:

getIncrements in interface HbaseEventSerializer

close

public void close()

Specified by:

close in interface HbaseEventSerializer