org.apache.flume.sink.hdfs
Class AbstractHDFSWriter

java.lang.Object
  org.apache.flume.sink.hdfs.AbstractHDFSWriter

All Implemented Interfaces:

Configurable, HDFSWriter

Direct Known Subclasses:

HDFSCompressedDataStream, HDFSDataStream, HDFSSequenceFile

@InterfaceAudience.Private
@InterfaceStability.Evolving
public abstract class AbstractHDFSWriter
extends Object
implements HDFSWriter

Constructor Summary

AbstractHDFSWriter()

Method Summary

void	configure(Context context) Request the implementing class to (re)configure itself.
int	getFsDesiredReplication()
int	getNumCurrentReplicas() This method gets the datanode replication count for the current open file.
boolean	isUnderReplicated() Contract for subclasses: Call registerCurrentStream() on open, unregisterCurrentStream() on close, and the base class takes care of the rest.
protected void	registerCurrentStream(org.apache.hadoop.fs.FSDataOutputStream outputStream, org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path destPath)
protected void	unregisterCurrentStream()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.flume.sink.hdfs.HDFSWriter

append, close, open, open, sync

Constructor Detail

AbstractHDFSWriter

public AbstractHDFSWriter()

Method Detail

configure

public void configure(Context context)

Description copied from interface: Configurable

Request the implementing class to (re)configure itself.

When configuration parameters are changed, they must be reflected by the component asap.

There are no thread safety guarrantees on when configure might be called.

Specified by:

configure in interface Configurable

isUnderReplicated

public boolean isUnderReplicated()

Contract for subclasses: Call registerCurrentStream() on open, unregisterCurrentStream() on close, and the base class takes care of the rest.

Specified by:

isUnderReplicated in interface HDFSWriter

Returns:

registerCurrentStream

protected void registerCurrentStream(org.apache.hadoop.fs.FSDataOutputStream outputStream,
                                     org.apache.hadoop.fs.FileSystem fs,
                                     org.apache.hadoop.fs.Path destPath)

unregisterCurrentStream

protected void unregisterCurrentStream()

getFsDesiredReplication

public int getFsDesiredReplication()

getNumCurrentReplicas

public int getNumCurrentReplicas()
                          throws IllegalArgumentException,
                                 IllegalAccessException,
                                 InvocationTargetException

This method gets the datanode replication count for the current open file. If the pipeline isn't started yet or is empty, you will get the default replication factor.

If this function returns -1, it means you are not properly running with the HDFS-826 patch.

Throws:

InvocationTargetException

IllegalAccessException

IllegalArgumentException