org.apache.avro.mapreduce

Class AvroKeyRecordWriter<T>

java.lang.Object

org.apache.hadoop.mapreduce.RecordWriter<AvroKey<T>,org.apache.hadoop.io.NullWritable>

org.apache.avro.mapreduce.AvroKeyRecordWriter<T>

Type Parameters:

T - The Java type of the Avro data to write.

All Implemented Interfaces:

Syncable

public class AvroKeyRecordWriter<T>
extends org.apache.hadoop.mapreduce.RecordWriter<AvroKey<T>,org.apache.hadoop.io.NullWritable>
implements Syncable

Writes Avro records to an Avro container file output stream.

Constructor Summary

Constructors

Constructor and Description
AvroKeyRecordWriter(Schema writerSchema, GenericData dataModel, CodecFactory compressionCodec, OutputStream outputStream) Constructor.
AvroKeyRecordWriter(Schema writerSchema, GenericData dataModel, CodecFactory compressionCodec, OutputStream outputStream, int syncInterval) Constructor.

Method Summary

Modifier and Type	Method and Description
void	close(org.apache.hadoop.mapreduce.TaskAttemptContext context)
long	sync() Return the current position as a value that may be passed to DataFileReader.seek(long).
void	write(AvroKey<T> record, org.apache.hadoop.io.NullWritable ignore)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

AvroKeyRecordWriter

public AvroKeyRecordWriter(Schema writerSchema,
                           GenericData dataModel,
                           CodecFactory compressionCodec,
                           OutputStream outputStream,
                           int syncInterval)
                    throws IOException

Constructor.

Parameters:

writerSchema - The writer schema for the records in the Avro container file.

compressionCodec - A compression codec factory for the Avro container file.

outputStream - The output stream to write the Avro container file to.

syncInterval - The sync interval for the Avro container file.

Throws:

IOException - If the record writer cannot be opened.

AvroKeyRecordWriter

public AvroKeyRecordWriter(Schema writerSchema,
                           GenericData dataModel,
                           CodecFactory compressionCodec,
                           OutputStream outputStream)
                    throws IOException

Constructor.

Parameters:

writerSchema - The writer schema for the records in the Avro container file.

compressionCodec - A compression codec factory for the Avro container file.

outputStream - The output stream to write the Avro container file to.

Throws:

IOException - If the record writer cannot be opened.

Method Detail

write

public void write(AvroKey<T> record,
                  org.apache.hadoop.io.NullWritable ignore)
           throws IOException

Specified by:

write in class org.apache.hadoop.mapreduce.RecordWriter<AvroKey<T>,org.apache.hadoop.io.NullWritable>

Throws:

IOException

close

public void close(org.apache.hadoop.mapreduce.TaskAttemptContext context)
           throws IOException

Specified by:

close in class org.apache.hadoop.mapreduce.RecordWriter<AvroKey<T>,org.apache.hadoop.io.NullWritable>

Throws:

IOException

sync

public long sync()
          throws IOException

Return the current position as a value that may be passed to DataFileReader.seek(long). Forces the end of the current block, emitting a synchronization marker.

Specified by:

sync in interface Syncable

Throws:

IOException - - if an error occurred while attempting to sync.