org.apache.flume.sink.hdfs
Class AbstractHDFSWriter

java.lang.Object
  extended by org.apache.flume.sink.hdfs.AbstractHDFSWriter
All Implemented Interfaces:
Configurable, HDFSWriter
Direct Known Subclasses:
HDFSCompressedDataStream, HDFSDataStream, HDFSSequenceFile

@InterfaceAudience.Private
@InterfaceStability.Evolving
public abstract class AbstractHDFSWriter
extends Object
implements HDFSWriter


Constructor Summary
AbstractHDFSWriter()
           
 
Method Summary
 void configure(Context context)
           Request the implementing class to (re)configure itself.
 int getFsDesiredReplication()
           
 int getNumCurrentReplicas()
          This method gets the datanode replication count for the current open file.
 boolean isUnderReplicated()
          Contract for subclasses: Call registerCurrentStream() on open, unregisterCurrentStream() on close, and the base class takes care of the rest.
protected  void registerCurrentStream(org.apache.hadoop.fs.FSDataOutputStream outputStream, org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path destPath)
           
protected  void unregisterCurrentStream()
           
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 
Methods inherited from interface org.apache.flume.sink.hdfs.HDFSWriter
append, close, open, open, sync
 

Constructor Detail

AbstractHDFSWriter

public AbstractHDFSWriter()
Method Detail

configure

public void configure(Context context)
Description copied from interface: Configurable

Request the implementing class to (re)configure itself.

When configuration parameters are changed, they must be reflected by the component asap.

There are no thread safety guarrantees on when configure might be called.

Specified by:
configure in interface Configurable

isUnderReplicated

public boolean isUnderReplicated()
Contract for subclasses: Call registerCurrentStream() on open, unregisterCurrentStream() on close, and the base class takes care of the rest.

Specified by:
isUnderReplicated in interface HDFSWriter
Returns:

registerCurrentStream

protected void registerCurrentStream(org.apache.hadoop.fs.FSDataOutputStream outputStream,
                                     org.apache.hadoop.fs.FileSystem fs,
                                     org.apache.hadoop.fs.Path destPath)

unregisterCurrentStream

protected void unregisterCurrentStream()

getFsDesiredReplication

public int getFsDesiredReplication()

getNumCurrentReplicas

public int getNumCurrentReplicas()
                          throws IllegalArgumentException,
                                 IllegalAccessException,
                                 InvocationTargetException
This method gets the datanode replication count for the current open file. If the pipeline isn't started yet or is empty, you will get the default replication factor.

If this function returns -1, it means you are not properly running with the HDFS-826 patch.

Throws:
InvocationTargetException
IllegalAccessException
IllegalArgumentException


Copyright © 2009-2013 Apache Software Foundation. All Rights Reserved.