org.apache.flume.channel.jdbc.impl
Class DerbySchemaHandler

java.lang.Object
  org.apache.flume.channel.jdbc.impl.DerbySchemaHandler

All Implemented Interfaces:

SchemaHandler

public class DerbySchemaHandler
extends Object
implements SchemaHandler

Schema handler for Derby Database. This handler uses the following schema:

FL_EVENT: The main event table. This table contains an auto-generated event ID along with the first 16kb of payload data. If the payload is larger than 16kb, a spill indicator flag is set and the remaining data is recorded in the FL_PLSPILL table.

 +-------------------------------+
 | FL_EVENT                      |
 +-------------------------------+
 | FLE_ID     : BIGINT PK        | (auto-gen sequence)
 | FLE_PAYLOAD: VARBINARY(16384) | (16kb payload)
 | FLE_SPILL  : BOOLEAN          | (true if payload spills)
 | FLE_CHANNEL: VARCHAR(64)      |
 +-------------------------------+
 

FL_PLSPILL: This table holds payloads in excess of 16kb and relates back to the FL_EVENT table using foreign key reference via FLP_EVENT column.

 +---------------------+
 | FL_PLSPILL          |
 +---------------------+
 | FLP_EVENT  : BIGINT | (FK into FL_EVENT.FLE_ID)
 | FLP_SPILL  : BLOB   |
 +---------------------+
 

FL_HEADER: The table that holds headers. This table contains name value pairs of headers less than or up to first 255 bytes each. If a name is longer than 255 bytes, a spill indicator flag is set and the remaining bytes are recorded in FL_NMSPILL table. Similarly if the value is longer than 255 bytes, a spill indicator flag is set and the remaining bytes are recorded in FL_VLSPILL table. Each header record relates back to the FL_EVENT table using foreign key reference via FLH_EVENT column.

 +--------------------------+
 | FL_HEADER                |
 +--------------------------+
 | FLH_ID     : BIGINT PK   | (auto-gen sequence)
 | FLH_EVENT  : BIGINT      | (FK into FL_EVENT.FLE_ID)
 | FLH_NAME   : VARCHAR(251)|
 | FLH_VALUE  : VARCHAR(251)|
 | FLH_NMSPILL: BOOLEAN     | (true if name spills)
 | FLH_VLSPILL: BOOLEAN     | (true if value spills)
 +--------------------------+
 

FL_NMSPILL: The table that holds header names in excess of 255 bytes and relates back to the FL_HEADER table using foreign key reference via FLN_HEADER column.

 +----------------------+
 | FL_NMSPILL           |
 +----------------------+
 | FLN_HEADER  : BIGINT | (FK into FL_HEADER.FLH_ID)
 | FLN_SPILL   : CLOB   |
 +----------------------+
 

FL_VLSPILL: The table that holds header values in excess of 255 bytes and relates back to the FL_HEADER table using foreign key reference via FLV_HEADER column.

 +----------------------+
 | FL_VLSPILL           |
 +----------------------+
 | FLV_HEADER  : BIGINT | (FK into FL_HEADER.FLH_ID)
 | FLV_SPILL   : CLOB   |
 +----------------------+
 

NOTE: The values that decide the spill boundary and storage length limits are defined in ConfigurationConstants class.

See Also:

ConfigurationConstants

Field Summary

static String	COLUMN_LOOKUP_QUERY
static String	QUERY_CHANNEL_SIZE
static String	QUERY_CREATE_INDEX_FLE_CHANNEL
static String	QUERY_CREATE_INDEX_FLH_EVENT
static String	QUERY_CREATE_INDEX_FLN_HEADER
static String	QUERY_CREATE_INDEX_FLP_EVENT
static String	QUERY_CREATE_INDEX_FLV_HEADER
static String	QUERY_CREATE_SCHEMA_FLUME
static String	QUERY_CREATE_TABLE_FL_EVENT
static String	QUERY_CREATE_TABLE_FL_HEADER_FK
static String	QUERY_CREATE_TABLE_FL_HEADER_FMT
static String	QUERY_CREATE_TABLE_FL_HEADER_NOFK
static String	QUERY_CREATE_TABLE_FL_NMSPILL_FK
static String	QUERY_CREATE_TABLE_FL_NMSPILL_FMT
static String	QUERY_CREATE_TABLE_FL_NMSPILL_NOFK
static String	QUERY_CREATE_TABLE_FL_PLSPILL_FK
static String	QUERY_CREATE_TABLE_FL_PLSPILL_FMT
static String	QUERY_CREATE_TABLE_FL_PLSPILL_NOFK
static String	QUERY_CREATE_TABLE_FL_VLSPILL_FK
static String	QUERY_CREATE_TABLE_FL_VLSPILL_FMT
static String	QUERY_CREATE_TABLE_FL_VLSPILL_NOFK
static String	STMT_DELETE_EVENT_BASE
static String	STMT_DELETE_EVENT_SPILL
static String	STMT_DELETE_HEADER_BASE
static String	STMT_DELETE_HEADER_NAME_SPILL
static String	STMT_DELETE_HEADER_VALUE_SPILL
static String	STMT_FETCH_HEADER_BASE
static String	STMT_FETCH_HEADER_NAME_SPILL
static String	STMT_FETCH_HEADER_VALUE_SPILL
static String	STMT_FETCH_PAYLOAD_BASE
static String	STMT_FETCH_PAYLOAD_SPILL
static String	STMT_INSERT_EVENT_BASE
static String	STMT_INSERT_EVENT_SPILL
static String	STMT_INSERT_HEADER_BASE
static String	STMT_INSERT_HEADER_NAME_SPILL
static String	STMT_INSERT_HEADER_VALUE_SPILL

Constructor Summary

protected

DerbySchemaHandler(DataSource dataSource)

Method Summary

void	createSchemaObjects(boolean createForeignKeys, boolean createIndex) Creates the schema.
PersistableEvent	fetchAndDeleteEvent(String channel, Connection connection) Retrieves the next persistent event from the database.
long	getChannelSize(Connection connection) Returns the current size of the channel using the connection specified that must have an active transaction ongoing.
boolean	schemaExists()
void	storeEvent(PersistableEvent pe, Connection connection) Inserts the given persistent event into the database.
void	validateSchema() Validates the schema.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

QUERY_CREATE_SCHEMA_FLUME

public static final String QUERY_CREATE_SCHEMA_FLUME

See Also:

Constant Field Values

QUERY_CREATE_TABLE_FL_EVENT

public static final String QUERY_CREATE_TABLE_FL_EVENT

QUERY_CREATE_INDEX_FLE_CHANNEL

public static final String QUERY_CREATE_INDEX_FLE_CHANNEL

See Also:

Constant Field Values

QUERY_CREATE_TABLE_FL_PLSPILL_FMT

public static final String QUERY_CREATE_TABLE_FL_PLSPILL_FMT

See Also:

Constant Field Values

QUERY_CREATE_TABLE_FL_PLSPILL_FK

public static final String QUERY_CREATE_TABLE_FL_PLSPILL_FK

QUERY_CREATE_TABLE_FL_PLSPILL_NOFK

public static final String QUERY_CREATE_TABLE_FL_PLSPILL_NOFK

QUERY_CREATE_INDEX_FLP_EVENT

public static final String QUERY_CREATE_INDEX_FLP_EVENT

See Also:

Constant Field Values

QUERY_CREATE_TABLE_FL_HEADER_FMT

public static final String QUERY_CREATE_TABLE_FL_HEADER_FMT

QUERY_CREATE_TABLE_FL_HEADER_FK

public static final String QUERY_CREATE_TABLE_FL_HEADER_FK

QUERY_CREATE_TABLE_FL_HEADER_NOFK

public static final String QUERY_CREATE_TABLE_FL_HEADER_NOFK

QUERY_CREATE_INDEX_FLH_EVENT

public static final String QUERY_CREATE_INDEX_FLH_EVENT

See Also:

Constant Field Values

QUERY_CREATE_TABLE_FL_NMSPILL_FMT

public static final String QUERY_CREATE_TABLE_FL_NMSPILL_FMT

QUERY_CREATE_TABLE_FL_NMSPILL_FK

public static final String QUERY_CREATE_TABLE_FL_NMSPILL_FK

QUERY_CREATE_TABLE_FL_NMSPILL_NOFK

public static final String QUERY_CREATE_TABLE_FL_NMSPILL_NOFK

QUERY_CREATE_INDEX_FLN_HEADER

public static final String QUERY_CREATE_INDEX_FLN_HEADER

See Also:

Constant Field Values

QUERY_CREATE_TABLE_FL_VLSPILL_FMT

public static final String QUERY_CREATE_TABLE_FL_VLSPILL_FMT

QUERY_CREATE_TABLE_FL_VLSPILL_FK

public static final String QUERY_CREATE_TABLE_FL_VLSPILL_FK

QUERY_CREATE_TABLE_FL_VLSPILL_NOFK

public static final String QUERY_CREATE_TABLE_FL_VLSPILL_NOFK

QUERY_CREATE_INDEX_FLV_HEADER

public static final String QUERY_CREATE_INDEX_FLV_HEADER

See Also:

Constant Field Values

COLUMN_LOOKUP_QUERY

public static final String COLUMN_LOOKUP_QUERY

See Also:

Constant Field Values

QUERY_CHANNEL_SIZE

public static final String QUERY_CHANNEL_SIZE

See Also:

Constant Field Values

STMT_INSERT_EVENT_BASE

public static final String STMT_INSERT_EVENT_BASE

See Also:

Constant Field Values

STMT_INSERT_EVENT_SPILL

public static final String STMT_INSERT_EVENT_SPILL

See Also:

Constant Field Values

STMT_INSERT_HEADER_BASE

public static final String STMT_INSERT_HEADER_BASE

See Also:

Constant Field Values

STMT_INSERT_HEADER_NAME_SPILL

public static final String STMT_INSERT_HEADER_NAME_SPILL

See Also:

Constant Field Values

STMT_INSERT_HEADER_VALUE_SPILL

public static final String STMT_INSERT_HEADER_VALUE_SPILL

See Also:

Constant Field Values

STMT_FETCH_PAYLOAD_BASE

public static final String STMT_FETCH_PAYLOAD_BASE

See Also:

Constant Field Values

STMT_FETCH_PAYLOAD_SPILL

public static final String STMT_FETCH_PAYLOAD_SPILL

See Also:

Constant Field Values

STMT_FETCH_HEADER_BASE

public static final String STMT_FETCH_HEADER_BASE

See Also:

Constant Field Values

STMT_FETCH_HEADER_NAME_SPILL

public static final String STMT_FETCH_HEADER_NAME_SPILL

See Also:

Constant Field Values

STMT_FETCH_HEADER_VALUE_SPILL

public static final String STMT_FETCH_HEADER_VALUE_SPILL

See Also:

Constant Field Values

STMT_DELETE_HEADER_VALUE_SPILL

public static final String STMT_DELETE_HEADER_VALUE_SPILL

See Also:

Constant Field Values

STMT_DELETE_HEADER_NAME_SPILL

public static final String STMT_DELETE_HEADER_NAME_SPILL

See Also:

Constant Field Values

STMT_DELETE_EVENT_SPILL

public static final String STMT_DELETE_EVENT_SPILL

See Also:

Constant Field Values

STMT_DELETE_HEADER_BASE

public static final String STMT_DELETE_HEADER_BASE

See Also:

Constant Field Values

STMT_DELETE_EVENT_BASE

public static final String STMT_DELETE_EVENT_BASE

See Also:

Constant Field Values

Constructor Detail

DerbySchemaHandler

protected DerbySchemaHandler(DataSource dataSource)

Method Detail

schemaExists

public boolean schemaExists()

Specified by:

schemaExists in interface SchemaHandler

Returns:

true if the schema exists. False otherwise.

createSchemaObjects

public void createSchemaObjects(boolean createForeignKeys,
                                boolean createIndex)

Description copied from interface: SchemaHandler

Creates the schema.

Specified by:

createSchemaObjects in interface SchemaHandler

Parameters:

createForeignKeys - a flag which indicates if the foreign key constraints should be created where necessary.

createIndex - a flag which indicates if indexes must be created during the creation of the schema.

validateSchema

public void validateSchema()

Description copied from interface: SchemaHandler

Validates the schema.

Specified by:

validateSchema in interface SchemaHandler

storeEvent

public void storeEvent(PersistableEvent pe,
                       Connection connection)

Description copied from interface: SchemaHandler

Inserts the given persistent event into the database. The connection that is passed into the handler has an ongoing transaction and therefore the SchemaHandler implementation must not close the connection.

Specified by:

storeEvent in interface SchemaHandler

Parameters:

pe - the event to persist

connection - the connection to use

fetchAndDeleteEvent

public PersistableEvent fetchAndDeleteEvent(String channel,
                                            Connection connection)

Description copied from interface: SchemaHandler

Retrieves the next persistent event from the database. The connection that is passed into the handler has an ongoing transaction and therefore the SchemaHandler implementation must not close the connection.

Specified by:

fetchAndDeleteEvent in interface SchemaHandler

Parameters:

channel - the channel name from which event will be retrieved

connection - the connection to use

Returns:

the next persistent event if available or null

getChannelSize

public long getChannelSize(Connection connection)

Description copied from interface: SchemaHandler

Returns the current size of the channel using the connection specified that must have an active transaction ongoing. This allows the provider impl to enforce channel capacity limits when persisting events.

Specified by:

getChannelSize in interface SchemaHandler

Returns:

the current size of the channel.