Class PropertiesFileConfigurationProvider

  extended by org.apache.flume.node.AbstractConfigurationProvider
      extended by org.apache.flume.node.PropertiesFileConfigurationProvider
All Implemented Interfaces:
Direct Known Subclasses:

public class PropertiesFileConfigurationProvider
extends AbstractConfigurationProvider

A configuration provider that uses properties file for specifying configuration. The configuration files follow the Java properties file syntax rules specified at Properties.load( Every configuration value specified in the properties file is prefixed by an Agent Name which helps isolate an individual agent's namespace.

Valid configuration files must observe the following rules for every agent namespace.

Apart from the above required configuration values, each source, sink or channel can have its own set of arbitrary configuration as required by the implementation. Each of these configuration values are expressed by fully namespace qualified configuration keys. For example, the configuration property called capacity for a channel called ch1 for the agent named host1 with value 1000 will be expressed as: host1.channels.ch1.capacity = 1000.

Any information contained in the configuration file other than what pertains to the configured agents, sources, sinks and channels via the explicitly enumerated list of sources, sinks and channels per agent name are ignored by this provider. Moreover, if any of the required configuration values are not present in the configuration file for the configured entities, that entity and anything that depends upon it is considered invalid and consequently not configured. For example, if a channel is missing its type attribute, it is considered misconfigured. Also, any sources or sinks that depend upon this channel are also considered misconfigured and not initialized.

Example configuration file:

 # Flume Configuration
 # This file contains configuration for one Agent identified as host1.

 host1.sources = avroSource thriftSource
 host1.channels = jdbcChannel
 host1.sinks = hdfsSink

 # avroSource configuration
 host1.sources.avroSource.type = org.apache.flume.source.AvroSource
 host1.sources.avroSource.runner.type = avro
 host1.sources.avroSource.runner.port = 11001
 host1.sources.avroSource.channels = jdbcChannel
 host1.sources.avroSource.selector.type = replicating

 # thriftSource configuration
 host1.sources.thriftSource.type = org.apache.flume.source.ThriftSource
 host1.sources.thriftSource.runner.type = thrift
 host1.sources.thriftSource.runner.port = 12001
 host1.sources.thriftSource.channels = jdbcChannel

 # jdbcChannel configuration
 host1.channels.jdbcChannel.type = jdbc
 host1.channels.jdbcChannel.jdbc.driver = com.mysql.jdbc.Driver
 host1.channels.jdbcChannel.jdbc.connect.url = http://localhost/flumedb
 host1.channels.jdbcChannel.jdbc.username = flume
 host1.channels.jdbcChannel.jdbc.password = flume

 # hdfsSink configuration
 host1.sinks.hdfsSink.type = hdfs
 host1.sinks.hdfsSink.hdfs.path = hdfs://localhost/
 host1.sinks.hdfsSink.batchsize = 1000
 host1.sinks.hdfsSink.runner.type = polling
 host1.sinks.hdfsSink.runner.polling.interval = 60

See Also:

Constructor Summary
PropertiesFileConfigurationProvider(String agentName, File file)
Method Summary
 FlumeConfiguration getFlumeConfiguration()
Methods inherited from class org.apache.flume.node.AbstractConfigurationProvider
getAgentName, getConfiguration
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail


public PropertiesFileConfigurationProvider(String agentName,
                                           File file)
Method Detail


public FlumeConfiguration getFlumeConfiguration()
Specified by:
getFlumeConfiguration in class AbstractConfigurationProvider

Copyright © 2009-2014 Apache Software Foundation. All Rights Reserved.