org.apache.flume.sink.solr.morphline

Class BlobDeserializer

java.lang.Object

org.apache.flume.sink.solr.morphline.BlobDeserializer

All Implemented Interfaces:

Closeable, AutoCloseable, EventDeserializer, Resettable

@InterfaceAudience.Private
@InterfaceStability.Evolving
public class BlobDeserializer
extends Object
implements EventDeserializer

A deserializer that reads a Binary Large Object (BLOB) per event, typically one BLOB per file; To be used in conjunction with Flume SpoolDirectorySource.

Note that this approach is not suitable for very large objects because it buffers up the entire BLOB.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	BlobDeserializer.Builder Builder implementations MUST have a public no-arg constructor

Field Summary

Fields

Modifier and Type	Field and Description
static int	MAX_BLOB_LENGTH_DEFAULT
static String	MAX_BLOB_LENGTH_KEY

Constructor Summary

Constructors

Modifier	Constructor and Description
protected	BlobDeserializer(Context context, ResettableInputStream in)

Method Summary

Methods

Modifier and Type	Method and Description
void	close() Calls EventDeserializer.reset() on the stream and then closes it.
void	mark() Marks the underlying input stream, indicating that the events previously returned by this EventDeserializer have been successfully committed.
Event	readEvent() Reads a BLOB from a file and returns an event
List<Event>	readEvents(int numEvents) Batch BLOB read
void	reset() Resets the underlying input stream to the last known mark (or beginning of the stream if EventDeserializer.mark() was never previously called.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

MAX_BLOB_LENGTH_KEY

public static final String MAX_BLOB_LENGTH_KEY

See Also:

Constant Field Values

MAX_BLOB_LENGTH_DEFAULT

public static final int MAX_BLOB_LENGTH_DEFAULT

See Also:

Constant Field Values

Constructor Detail

BlobDeserializer

protected BlobDeserializer(Context context,
                ResettableInputStream in)

Method Detail

readEvent

public Event readEvent()
                throws IOException

Reads a BLOB from a file and returns an event

Specified by:

readEvent in interface EventDeserializer

Returns:

Event containing a BLOB

Throws:

IOException

See Also:

EventDeserializer.mark(), EventDeserializer.reset()

readEvents

public List<Event> readEvents(int numEvents)
                       throws IOException

Batch BLOB read

Specified by:

readEvents in interface EventDeserializer

Parameters:

numEvents - Maximum number of events to return.

Returns:

List of events containing read BLOBs

Throws:

IOException

See Also:

EventDeserializer.mark(), EventDeserializer.reset()

mark

public void mark()
          throws IOException

Description copied from interface: EventDeserializer

Marks the underlying input stream, indicating that the events previously returned by this EventDeserializer have been successfully committed.

Specified by:

mark in interface EventDeserializer

Specified by:

mark in interface Resettable

Throws:

IOException

See Also:

EventDeserializer.reset()

reset

public void reset()
           throws IOException

Description copied from interface: EventDeserializer

Resets the underlying input stream to the last known mark (or beginning of the stream if EventDeserializer.mark() was never previously called. This should be done in the case of inability to commit previously-deserialized events.

Specified by:

reset in interface EventDeserializer

Specified by:

reset in interface Resettable

Throws:

IOException

See Also:

EventDeserializer.mark()

close

public void close()
           throws IOException

Description copied from interface: EventDeserializer

Calls EventDeserializer.reset() on the stream and then closes it. In the case of successful completion of event consumption, EventDeserializer.mark() MUST be called before close().

Specified by:

close in interface Closeable

Specified by:

close in interface AutoCloseable

Specified by:

close in interface EventDeserializer

Throws:

IOException

See Also:

EventDeserializer.mark(), EventDeserializer.reset()