T - The (java) type of the Trevni data to write.
Writes a directory of files per task, each comprising a single filesystem block. To reduce the number of files, increase the default filesystem block size for the job. Each task also requires enough memory to buffer a filesystem block.
public class AvroTrevniKeyOutputFormat<T> extends FileOutputFormat<AvroKey<T>,NullWritable>
OutputFormat that writes Avro data to
Trevni files.
This implement was modeled off
AvroKeyOutputFormat to allow for easy
transition
FileOutputFormat for writing Trevni container files.
Since Trevni container files only contain records (not key/value pairs), this output format ignores the value.
FileOutputFormat.CounterBASE_OUTPUT_NAME, COMPRESS, COMPRESS_CODEC, COMPRESS_TYPE, OUTDIR, PART| Constructor and Description |
|---|
AvroTrevniKeyOutputFormat() |
| Modifier and Type | Method and Description |
|---|---|
RecordWriter<AvroKey<T>,NullWritable> |
getRecordWriter(TaskAttemptContext context) |
checkOutputSpecs, getCompressOutput, getDefaultWorkFile, getOutputCommitter, getOutputCompressorClass, getOutputName, getOutputPath, getPathForWorkFile, getUniqueFile, getWorkOutputPath, setCompressOutput, setOutputCompressorClass, setOutputName, setOutputPathpublic RecordWriter<AvroKey<T>,NullWritable> getRecordWriter(TaskAttemptContext context) throws IOException, InterruptedException
getRecordWriter in class FileOutputFormat<AvroKey<T>,NullWritable>IOExceptionInterruptedExceptionCopyright © 2009–2020 The Apache Software Foundation. All rights reserved.