Package org.apache.trevni.avro.mapreduce
Class AvroTrevniKeyOutputFormat<T>
java.lang.Object
org.apache.hadoop.mapreduce.OutputFormat<AvroKey<T>,NullWritable>
org.apache.hadoop.mapreduce.lib.output.FileOutputFormat<AvroKey<T>,NullWritable>
org.apache.trevni.avro.mapreduce.AvroTrevniKeyOutputFormat<T>
- Type Parameters:
T
- The (java) type of the Trevni data to write.Writes a directory of files per task, each comprising a single filesystem block. To reduce the number of files, increase the default filesystem block size for the job. Each task also requires enough memory to buffer a filesystem block.
An
OutputFormat
that writes Avro data to
Trevni files.
This implement was modeled off
AvroKeyOutputFormat
to allow for easy
transition
FileOutputFormat for writing Trevni container files.
Since Trevni container files only contain records (not key/value pairs), this output format ignores the value.
-
Nested Class Summary
Nested classes/interfaces inherited from class org.apache.hadoop.mapreduce.lib.output.FileOutputFormat
FileOutputFormat.Counter
-
Field Summary
Fields inherited from class org.apache.hadoop.mapreduce.lib.output.FileOutputFormat
BASE_OUTPUT_NAME, COMPRESS, COMPRESS_CODEC, COMPRESS_TYPE, OUTDIR, PART
-
Constructor Summary
-
Method Summary
Methods inherited from class org.apache.hadoop.mapreduce.lib.output.FileOutputFormat
checkOutputSpecs, getCompressOutput, getDefaultWorkFile, getOutputCommitter, getOutputCompressorClass, getOutputName, getOutputPath, getPathForWorkFile, getUniqueFile, getWorkOutputPath, setCompressOutput, setOutputCompressorClass, setOutputName, setOutputPath
-
Constructor Details
-
AvroTrevniKeyOutputFormat
public AvroTrevniKeyOutputFormat()
-
-
Method Details
-
getRecordWriter
public RecordWriter<AvroKey<T>,NullWritable> getRecordWriter(TaskAttemptContext context) throws IOException, InterruptedException - Specified by:
getRecordWriter
in classFileOutputFormat<AvroKey<T>,
NullWritable> - Throws:
IOException
InterruptedException
-