Package org.apache.avro.mapred
Class AvroOutputFormat<T>
java.lang.Object
org.apache.hadoop.mapred.FileOutputFormat<AvroWrapper<T>,NullWritable>
org.apache.avro.mapred.AvroOutputFormat<T>
- All Implemented Interfaces:
OutputFormat<AvroWrapper<T>,
NullWritable>
An
OutputFormat
for Avro data files.
You can specify various options using Job Configuration properties. Look at
the fields in AvroJob
as well as this class to get an overview of the
supported options.-
Nested Class Summary
Nested classes/interfaces inherited from class org.apache.hadoop.mapred.FileOutputFormat
FileOutputFormat.Counter
-
Field Summary
Modifier and TypeFieldDescriptionstatic final String
The configuration key for Avro deflate level.static final String
The file name extension for avro data files.static final String
The configuration key for Avro sync interval.static final String
The configuration key for Avro XZ level.static final String
The configuration key for Avro ZSTD buffer pool.static final String
The configuration key for Avro ZSTD level. -
Constructor Summary
-
Method Summary
Modifier and TypeMethodDescriptiongetRecordWriter
(FileSystem ignore, JobConf job, String name, Progressable prog) static void
setDeflateLevel
(JobConf job, int level) Enable output compression using the deflate codec and specify its level.static void
setSyncInterval
(JobConf job, int syncIntervalInBytes) Set the sync interval to be used by the underlyingDataFileWriter
.Methods inherited from class org.apache.hadoop.mapred.FileOutputFormat
checkOutputSpecs, getCompressOutput, getOutputCompressorClass, getOutputPath, getPathForCustomFile, getTaskOutputPath, getUniqueName, getWorkOutputPath, setCompressOutput, setOutputCompressorClass, setOutputPath, setWorkOutputPath
-
Field Details
-
EXT
The file name extension for avro data files.- See Also:
-
DEFLATE_LEVEL_KEY
The configuration key for Avro deflate level.- See Also:
-
XZ_LEVEL_KEY
The configuration key for Avro XZ level.- See Also:
-
ZSTD_LEVEL_KEY
The configuration key for Avro ZSTD level.- See Also:
-
ZSTD_BUFFERPOOL_KEY
The configuration key for Avro ZSTD buffer pool.- See Also:
-
SYNC_INTERVAL_KEY
The configuration key for Avro sync interval.- See Also:
-
-
Constructor Details
-
AvroOutputFormat
public AvroOutputFormat()
-
-
Method Details
-
setDeflateLevel
Enable output compression using the deflate codec and specify its level. -
setSyncInterval
Set the sync interval to be used by the underlyingDataFileWriter
. -
getRecordWriter
public RecordWriter<AvroWrapper<T>,NullWritable> getRecordWriter(FileSystem ignore, JobConf job, String name, Progressable prog) throws IOException - Specified by:
getRecordWriter
in interfaceOutputFormat<AvroWrapper<T>,
NullWritable> - Specified by:
getRecordWriter
in classFileOutputFormat<AvroWrapper<T>,
NullWritable> - Throws:
IOException
-