K
- The type of key the record writer should generate.V
- The type of value the record wrtier should generate.T
- The type of the entries within the Trevni container file being writen.public abstract class AvroTrevniRecordWriterBase<K,V,T> extends RecordWriter<K,V>
RecordWriter
s that writes Trevni container files.Modifier and Type | Field and Description |
---|---|
static String |
EXT
trevni file extension
|
protected ColumnFileMetaData |
meta
meta data to be stored in the output file.
|
static String |
META_PREFIX
prefix of job configs that we care about
|
protected int |
part
Counter that increments as new trevni files are create because the current file
has exceeded the block size
|
protected Schema |
schema
Provided avro schema from the context
|
protected AvroColumnWriter<T> |
writer
Trevni file writer
|
Constructor and Description |
---|
AvroTrevniRecordWriterBase(TaskAttemptContext context)
Constructor.
|
Modifier and Type | Method and Description |
---|---|
void |
close(TaskAttemptContext arg0) |
void |
flush()
A Trevni flush will close the current file and prep a new writer
|
protected abstract Schema |
initSchema(TaskAttemptContext context)
Use the task context to construct a schema for writing
|
write
public static final String EXT
public static final String META_PREFIX
protected int part
protected AvroColumnWriter<T> writer
protected Schema schema
protected ColumnFileMetaData meta
public AvroTrevniRecordWriterBase(TaskAttemptContext context) throws IOException
context
- The TaskAttempContext to supply the writer with information form the job configurationIOException
protected abstract Schema initSchema(TaskAttemptContext context)
IOException
public void flush() throws IOException
IOException
public void close(TaskAttemptContext arg0) throws IOException, InterruptedException
close
in class RecordWriter<K,V>
IOException
InterruptedException
Copyright © 2009–2017 The Apache Software Foundation. All rights reserved.