K
- The type of key to write.V
- The type of value to write.public class AvroKeyValueRecordWriter<K,V> extends RecordWriter<K,V> implements Syncable
Each entry in the Avro container file will be a generic record with two fields, named 'key' and 'value'. The input types may be basic Writable objects like Text or IntWritable, or they may be AvroWrapper subclasses (AvroKey or AvroValue). Writable objects will be converted to their corresponding Avro types when written to the generic record key/value pair.
Constructor and Description |
---|
AvroKeyValueRecordWriter(AvroDatumConverter<K,?> keyConverter,
AvroDatumConverter<V,?> valueConverter,
GenericData dataModel,
CodecFactory compressionCodec,
OutputStream outputStream)
Constructor.
|
AvroKeyValueRecordWriter(AvroDatumConverter<K,?> keyConverter,
AvroDatumConverter<V,?> valueConverter,
GenericData dataModel,
CodecFactory compressionCodec,
OutputStream outputStream,
int syncInterval)
Constructor.
|
Modifier and Type | Method and Description |
---|---|
void |
close(TaskAttemptContext context) |
Schema |
getWriterSchema()
Gets the writer schema for the key/value pair generic record.
|
long |
sync()
Return the current position as a value that may be passed to DataFileReader.seek(long).
|
void |
write(K key,
V value) |
public AvroKeyValueRecordWriter(AvroDatumConverter<K,?> keyConverter, AvroDatumConverter<V,?> valueConverter, GenericData dataModel, CodecFactory compressionCodec, OutputStream outputStream, int syncInterval) throws IOException
keyConverter
- A key to Avro datum converter.valueConverter
- A value to Avro datum converter.dataModel
- The data model for key and value.compressionCodec
- A compression codec factory for the Avro container file.outputStream
- The output stream to write the Avro container file to.syncInterval
- The sync interval for the Avro container file.IOException
- If the record writer cannot be opened.public AvroKeyValueRecordWriter(AvroDatumConverter<K,?> keyConverter, AvroDatumConverter<V,?> valueConverter, GenericData dataModel, CodecFactory compressionCodec, OutputStream outputStream) throws IOException
keyConverter
- A key to Avro datum converter.valueConverter
- A value to Avro datum converter.dataModel
- The data model for key and value.compressionCodec
- A compression codec factory for the Avro container file.outputStream
- The output stream to write the Avro container file to.IOException
- If the record writer cannot be opened.public Schema getWriterSchema()
public void write(K key, V value) throws IOException
write
in class RecordWriter<K,V>
IOException
public void close(TaskAttemptContext context) throws IOException
close
in class RecordWriter<K,V>
IOException
public long sync() throws IOException
sync
in interface Syncable
IOException
- - if an error occurred while attempting to sync.Copyright © 2009-2014 The Apache Software Foundation. All Rights Reserved.