org.apache.flume.client.avro
Class ReliableSpoolingFileEventReader

java.lang.Object
  extended by org.apache.flume.client.avro.ReliableSpoolingFileEventReader
All Implemented Interfaces:
Closeable, EventReader, ReliableEventReader

@InterfaceAudience.Private
@InterfaceStability.Evolving
public class ReliableSpoolingFileEventReader
extends Object
implements ReliableEventReader

A ReliableEventReader which reads log data from files stored in a spooling directory and renames each file once all of its data has been read (through EventDeserializer.readEvent() calls). The user must commit() each read, to indicate that the lines have been fully processed.

Read calls will return no data if there are no files left to read. This class, in general, is not thread safe.

This reader assumes that files with unique file names are left in the spooling directory and not modified once they are placed there. Any user behavior which violates these assumptions, when detected, will result in a FlumeException being thrown.

This class makes the following guarantees, if above assumptions are met:


Nested Class Summary
static class ReliableSpoolingFileEventReader.Builder
          Special builder class for ReliableSpoolingFileEventReader
 
Method Summary
 void close()
          Clean-up any state associated with this reader.
 void commit()
          Commit the last lines which were read.
 String getLastFileRead()
          Return the filename which generated the data from the last successful readEvents(int) call.
 Event readEvent()
          Get the next line associated with the input stream.
 List<Event> readEvents(int numEvents)
          Get up to n lines associated with the input stream.
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Method Detail

getLastFileRead

public String getLastFileRead()
Return the filename which generated the data from the last successful readEvents(int) call. Returns null if called before any file contents are read.


readEvent

public Event readEvent()
                throws IOException
Description copied from interface: EventReader
Get the next line associated with the input stream. If this returns null, the input underlying input source is considered finished. Note that this is allowed to block for indefinite amounts of time waiting to generate a new line.

Specified by:
readEvent in interface EventReader
Throws:
IOException

readEvents

public List<Event> readEvents(int numEvents)
                       throws IOException
Description copied from interface: EventReader
Get up to n lines associated with the input stream. If this returns less than n lines, the input underlying input source is considered finished. Note that this is allowed to block for indefinite amounts of time waiting to generate a new line.

Specified by:
readEvents in interface EventReader
Throws:
IOException

close

public void close()
           throws IOException
Description copied from interface: EventReader
Clean-up any state associated with this reader.

Specified by:
close in interface Closeable
Specified by:
close in interface EventReader
Throws:
IOException

commit

public void commit()
            throws IOException
Commit the last lines which were read.

Specified by:
commit in interface ReliableEventReader
Throws:
IOException


Copyright © 2009-2015 Apache Software Foundation. All Rights Reserved.