org.apache.avro.mapred

Class AvroOutputFormat<T>

java.lang.Object

org.apache.hadoop.mapred.FileOutputFormat<AvroWrapper<T>,org.apache.hadoop.io.NullWritable>

org.apache.avro.mapred.AvroOutputFormat<T>

All Implemented Interfaces:

org.apache.hadoop.mapred.OutputFormat<AvroWrapper<T>,org.apache.hadoop.io.NullWritable>

public class AvroOutputFormat<T>
extends org.apache.hadoop.mapred.FileOutputFormat<AvroWrapper<T>,org.apache.hadoop.io.NullWritable>

An OutputFormat for Avro data files.

You can specify various options using Job Configuration properties. Look at the fields in AvroJob as well as this class to get an overview of the supported options.

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.hadoop.mapred.FileOutputFormat

org.apache.hadoop.mapred.FileOutputFormat.Counter

Field Summary

Fields

Modifier and Type	Field and Description
static int	DEFAULT_DEFLATE_LEVEL The default deflate level.
static String	DEFLATE_LEVEL_KEY The configuration key for Avro deflate level.
static String	EXT The file name extension for avro data files.
static String	SYNC_INTERVAL_KEY The configuration key for Avro sync interval.

Constructor Summary

Constructors

Constructor and Description
AvroOutputFormat()

Method Summary

Methods

Modifier and Type	Method and Description
org.apache.hadoop.mapred.RecordWriter<AvroWrapper<T>,org.apache.hadoop.io.NullWritable>	getRecordWriter(org.apache.hadoop.fs.FileSystem ignore, org.apache.hadoop.mapred.JobConf job, String name, org.apache.hadoop.util.Progressable prog)
static void	setDeflateLevel(org.apache.hadoop.mapred.JobConf job, int level) Enable output compression using the deflate codec and specify its level.
static void	setSyncInterval(org.apache.hadoop.mapred.JobConf job, int syncIntervalInBytes) Set the sync interval to be used by the underlying DataFileWriter.

Methods inherited from class org.apache.hadoop.mapred.FileOutputFormat

checkOutputSpecs, getCompressOutput, getOutputCompressorClass, getOutputPath, getPathForCustomFile, getTaskOutputPath, getUniqueName, getWorkOutputPath, setCompressOutput, setOutputCompressorClass, setOutputPath

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

EXT

public static final String EXT

The file name extension for avro data files.

See Also:

Constant Field Values

DEFLATE_LEVEL_KEY

public static final String DEFLATE_LEVEL_KEY

The configuration key for Avro deflate level.

See Also:

Constant Field Values

SYNC_INTERVAL_KEY

public static final String SYNC_INTERVAL_KEY

The configuration key for Avro sync interval.

See Also:

Constant Field Values

DEFAULT_DEFLATE_LEVEL

public static final int DEFAULT_DEFLATE_LEVEL

The default deflate level.

See Also:

Constant Field Values

Constructor Detail

AvroOutputFormat

public AvroOutputFormat()

Method Detail

setDeflateLevel

public static void setDeflateLevel(org.apache.hadoop.mapred.JobConf job,
                   int level)

Enable output compression using the deflate codec and specify its level.

setSyncInterval

public static void setSyncInterval(org.apache.hadoop.mapred.JobConf job,
                   int syncIntervalInBytes)

Set the sync interval to be used by the underlying DataFileWriter.

getRecordWriter

public org.apache.hadoop.mapred.RecordWriter<AvroWrapper<T>,org.apache.hadoop.io.NullWritable> getRecordWriter(org.apache.hadoop.fs.FileSystem ignore,
                                                                                                      org.apache.hadoop.mapred.JobConf job,
                                                                                                      String name,
                                                                                                      org.apache.hadoop.util.Progressable prog)
                                                                                                        throws IOException

Specified by:

getRecordWriter in interface org.apache.hadoop.mapred.OutputFormat<AvroWrapper<T>,org.apache.hadoop.io.NullWritable>

Specified by:

getRecordWriter in class org.apache.hadoop.mapred.FileOutputFormat<AvroWrapper<T>,org.apache.hadoop.io.NullWritable>

Throws:

IOException