org.apache.flume.sink.kite
Class DatasetSinkConstants

java.lang.Object
  extended by org.apache.flume.sink.kite.DatasetSinkConstants

public class DatasetSinkConstants
extends Object


Field Summary
static String AUTH_KEYTAB
           
static String AUTH_PRINCIPAL
           
static String AUTH_PROXY_USER
          Hadoop authentication settings
static String[] AVAILABLE_PARSERS
           
static String[] AVAILABLE_POLICIES
           
static String AVRO_ENTITY_PARSER
          Built-in entity parsers
static String AVRO_SCHEMA_LITERAL_HEADER
          Headers where avro schema information is expected.
static String AVRO_SCHEMA_URL_HEADER
           
static String CONFIG_ENTITY_PARSER
          Parser used to parse Flume Events into Kite entities.
static String CONFIG_FAILURE_POLICY
          Policy used to handle non-recoverable failures.
static String CONFIG_FLUSHABLE_COMMIT_ON_BATCH
          Flag for committing the Flume transaction on each batch for Flushable datasets.
static String CONFIG_KITE_BATCH_SIZE
          Number of records to process from the incoming channel per call to process.
static String CONFIG_KITE_DATASET_NAME
          Name of the Kite Dataset to write into.
static String CONFIG_KITE_DATASET_NAMESPACE
          Namespace of the Kite Dataset to write into.
static String CONFIG_KITE_DATASET_URI
          URI of the Kite Dataset
static String CONFIG_KITE_ERROR_DATASET_URI
          The URI to write non-recoverable Flume events to in the case of an error.
static String CONFIG_KITE_REPO_URI
          URI of the Kite DatasetRepository.
static String CONFIG_KITE_ROLL_INTERVAL
          Maximum time to wait before finishing files.
static String CONFIG_SYNCABLE_SYNC_ON_BATCH
          Flag for syncing the DatasetWriter on each batch for Syncable datasets.
static long DEFAULT_BATCH_SIZE
           
static String DEFAULT_ENTITY_PARSER
           
static String DEFAULT_FAILURE_POLICY
           
static boolean DEFAULT_FLUSHABLE_COMMIT_ON_BATCH
           
static String DEFAULT_NAMESPACE
           
static int DEFAULT_ROLL_INTERVAL
           
static boolean DEFAULT_SYNCABLE_SYNC_ON_BATCH
           
static String RETRY_FAILURE_POLICY
          Retry non-recoverable Flume events.
static String SAVE_FAILURE_POLICY
          Write non-recoverable Flume events to a Kite dataset.
 
Constructor Summary
DatasetSinkConstants()
           
 
Method Summary
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Field Detail

CONFIG_KITE_DATASET_URI

public static final String CONFIG_KITE_DATASET_URI
URI of the Kite Dataset

See Also:
Constant Field Values

CONFIG_KITE_REPO_URI

public static final String CONFIG_KITE_REPO_URI
URI of the Kite DatasetRepository.

See Also:
Constant Field Values

CONFIG_KITE_DATASET_NAME

public static final String CONFIG_KITE_DATASET_NAME
Name of the Kite Dataset to write into.

See Also:
Constant Field Values

CONFIG_KITE_DATASET_NAMESPACE

public static final String CONFIG_KITE_DATASET_NAMESPACE
Namespace of the Kite Dataset to write into.

See Also:
Constant Field Values

DEFAULT_NAMESPACE

public static final String DEFAULT_NAMESPACE
See Also:
Constant Field Values

CONFIG_KITE_BATCH_SIZE

public static final String CONFIG_KITE_BATCH_SIZE
Number of records to process from the incoming channel per call to process.

See Also:
Constant Field Values

DEFAULT_BATCH_SIZE

public static long DEFAULT_BATCH_SIZE

CONFIG_KITE_ROLL_INTERVAL

public static final String CONFIG_KITE_ROLL_INTERVAL
Maximum time to wait before finishing files.

See Also:
Constant Field Values

DEFAULT_ROLL_INTERVAL

public static int DEFAULT_ROLL_INTERVAL

CONFIG_FLUSHABLE_COMMIT_ON_BATCH

public static final String CONFIG_FLUSHABLE_COMMIT_ON_BATCH
Flag for committing the Flume transaction on each batch for Flushable datasets. When set to false, Flume will only commit the transaction when roll interval has expired. Setting this to false requires enough space in the channel to handle all events delivered during the roll interval. Defaults to true.

See Also:
Constant Field Values

DEFAULT_FLUSHABLE_COMMIT_ON_BATCH

public static boolean DEFAULT_FLUSHABLE_COMMIT_ON_BATCH

CONFIG_SYNCABLE_SYNC_ON_BATCH

public static final String CONFIG_SYNCABLE_SYNC_ON_BATCH
Flag for syncing the DatasetWriter on each batch for Syncable datasets. Defaults to true.

See Also:
Constant Field Values

DEFAULT_SYNCABLE_SYNC_ON_BATCH

public static boolean DEFAULT_SYNCABLE_SYNC_ON_BATCH

CONFIG_ENTITY_PARSER

public static final String CONFIG_ENTITY_PARSER
Parser used to parse Flume Events into Kite entities.

See Also:
Constant Field Values

AVRO_ENTITY_PARSER

public static final String AVRO_ENTITY_PARSER
Built-in entity parsers

See Also:
Constant Field Values

DEFAULT_ENTITY_PARSER

public static final String DEFAULT_ENTITY_PARSER
See Also:
Constant Field Values

AVAILABLE_PARSERS

public static final String[] AVAILABLE_PARSERS

CONFIG_FAILURE_POLICY

public static final String CONFIG_FAILURE_POLICY
Policy used to handle non-recoverable failures.

See Also:
Constant Field Values

SAVE_FAILURE_POLICY

public static final String SAVE_FAILURE_POLICY
Write non-recoverable Flume events to a Kite dataset.

See Also:
Constant Field Values

CONFIG_KITE_ERROR_DATASET_URI

public static final String CONFIG_KITE_ERROR_DATASET_URI
The URI to write non-recoverable Flume events to in the case of an error. If the dataset doesn't exist, it will be created.

See Also:
Constant Field Values

RETRY_FAILURE_POLICY

public static final String RETRY_FAILURE_POLICY
Retry non-recoverable Flume events. This will lead to a never ending cycle of failure, but matches the previous default semantics of the DatasetSink.

See Also:
Constant Field Values

DEFAULT_FAILURE_POLICY

public static final String DEFAULT_FAILURE_POLICY
See Also:
Constant Field Values

AVAILABLE_POLICIES

public static final String[] AVAILABLE_POLICIES

AVRO_SCHEMA_LITERAL_HEADER

public static final String AVRO_SCHEMA_LITERAL_HEADER
Headers where avro schema information is expected.

See Also:
Constant Field Values

AVRO_SCHEMA_URL_HEADER

public static final String AVRO_SCHEMA_URL_HEADER
See Also:
Constant Field Values

AUTH_PROXY_USER

public static final String AUTH_PROXY_USER
Hadoop authentication settings

See Also:
Constant Field Values

AUTH_PRINCIPAL

public static final String AUTH_PRINCIPAL
See Also:
Constant Field Values

AUTH_KEYTAB

public static final String AUTH_KEYTAB
See Also:
Constant Field Values
Constructor Detail

DatasetSinkConstants

public DatasetSinkConstants()


Copyright © 2009-2015 Apache Software Foundation. All Rights Reserved.