org.apache.avro.mapred

Class AvroJob

java.lang.Object

org.apache.avro.mapred.AvroJob

public class AvroJob
extends Object

Setters to configure jobs for Avro data.

Field Summary

Fields

Modifier and Type	Field and Description
static String	BINARY_PREFIX The configuration key prefix for a binary output metadata.
static String	INPUT_IS_REFLECT The configuration key for reflection-based input representation.
static String	INPUT_SCHEMA The configuration key for a job's input schema.
static String	MAP_OUTPUT_IS_REFLECT The configuration key for reflection-based map output representation.
static String	MAP_OUTPUT_SCHEMA The configuration key for a job's intermediate schema.
static String	OUTPUT_CODEC The configuration key for a job's output compression codec.
static String	OUTPUT_SCHEMA The configuration key for a job's output schema.
static String	TEXT_PREFIX The configuration key prefix for a text output metadata.

Method Summary

Methods

Modifier and Type	Method and Description
static GenericData	createDataModel(Configuration conf)
static GenericData	createInputDataModel(Configuration conf)
static GenericData	createMapOutputDataModel(Configuration conf)
static Class<? extends GenericData>	getDataModelClass(Configuration conf) Return the job's data model implementation class.
static Schema	getInputSchema(Configuration job) Return a job's map input schema.
static Schema	getMapOutputSchema(Configuration job) Return a job's map output key schema.
static Schema	getOutputSchema(Configuration job) Return a job's output key schema.
static void	setCombinerClass(JobConf job, Class<? extends AvroReducer> c) Configure a job's combiner implementation.
static void	setDataModelClass(JobConf job, Class<? extends GenericData> modelClass) Configure a job's data model implementation class.
static void	setInputReflect(JobConf job) Indicate that a job's input data should use reflect representation.
static void	setInputSchema(JobConf job, Schema s) Configure a job's map input schema.
static void	setInputSequenceFile(JobConf job) Indicate that a job's input files are in SequenceFile format.
static void	setMapOutputReflect(JobConf job) Indicate that a job's map output data should use reflect representation.
static void	setMapOutputSchema(JobConf job, Schema s) Configure a job's map output schema.
static void	setMapperClass(JobConf job, Class<? extends AvroMapper> c) Configure a job's mapper implementation.
static void	setOutputCodec(JobConf job, String codec) Configure a job's output compression codec.
static void	setOutputMeta(JobConf job, String key, byte[] value) Add metadata to job output files.
static void	setOutputMeta(JobConf job, String key, long value) Add metadata to job output files.
static void	setOutputMeta(JobConf job, String key, String value) Add metadata to job output files.
static void	setOutputSchema(JobConf job, Schema s) Configure a job's output schema.
static void	setReducerClass(JobConf job, Class<? extends AvroReducer> c) Configure a job's reducer implementation.
static void	setReflect(JobConf job) Indicate that all a job's data should use the reflect representation.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

INPUT_SCHEMA

public static final String INPUT_SCHEMA

The configuration key for a job's input schema.

See Also:

Constant Field Values

MAP_OUTPUT_SCHEMA

public static final String MAP_OUTPUT_SCHEMA

The configuration key for a job's intermediate schema.

See Also:

Constant Field Values

OUTPUT_SCHEMA

public static final String OUTPUT_SCHEMA

The configuration key for a job's output schema.

See Also:

Constant Field Values

OUTPUT_CODEC

public static final String OUTPUT_CODEC

The configuration key for a job's output compression codec. This takes one of the strings registered in CodecFactory

See Also:

Constant Field Values

TEXT_PREFIX

public static final String TEXT_PREFIX

The configuration key prefix for a text output metadata.

See Also:

Constant Field Values

BINARY_PREFIX

public static final String BINARY_PREFIX

The configuration key prefix for a binary output metadata.

See Also:

Constant Field Values

INPUT_IS_REFLECT

public static final String INPUT_IS_REFLECT

The configuration key for reflection-based input representation.

See Also:

Constant Field Values

MAP_OUTPUT_IS_REFLECT

public static final String MAP_OUTPUT_IS_REFLECT

The configuration key for reflection-based map output representation.

See Also:

Constant Field Values

Method Detail

setInputSchema

public static void setInputSchema(JobConf job,
                  Schema s)

Configure a job's map input schema.

getInputSchema

public static Schema getInputSchema(Configuration job)

Return a job's map input schema.

setMapOutputSchema

public static void setMapOutputSchema(JobConf job,
                      Schema s)

Configure a job's map output schema. The map output schema defaults to the output schema and need only be specified when it differs. Thus must be a Pair schema.

getMapOutputSchema

public static Schema getMapOutputSchema(Configuration job)

Return a job's map output key schema.

setOutputSchema

public static void setOutputSchema(JobConf job,
                   Schema s)

Configure a job's output schema. Unless this is a map-only job, this must be a Pair schema.

setOutputCodec

public static void setOutputCodec(JobConf job,
                  String codec)

Configure a job's output compression codec.

setOutputMeta

public static void setOutputMeta(JobConf job,
                 String key,
                 String value)

Add metadata to job output files.

setOutputMeta

public static void setOutputMeta(JobConf job,
                 String key,
                 long value)

Add metadata to job output files.

setOutputMeta

public static void setOutputMeta(JobConf job,
                 String key,
                 byte[] value)

Add metadata to job output files.

setInputSequenceFile

public static void setInputSequenceFile(JobConf job)

Indicate that a job's input files are in SequenceFile format.

setReflect

public static void setReflect(JobConf job)

Indicate that all a job's data should use the reflect representation.

setInputReflect

public static void setInputReflect(JobConf job)

Indicate that a job's input data should use reflect representation.

setMapOutputReflect

public static void setMapOutputReflect(JobConf job)

Indicate that a job's map output data should use reflect representation.

getOutputSchema

public static Schema getOutputSchema(Configuration job)

Return a job's output key schema.

setMapperClass

public static void setMapperClass(JobConf job,
                  Class<? extends AvroMapper> c)

Configure a job's mapper implementation.

setCombinerClass

public static void setCombinerClass(JobConf job,
                    Class<? extends AvroReducer> c)

Configure a job's combiner implementation.

setReducerClass

public static void setReducerClass(JobConf job,
                   Class<? extends AvroReducer> c)

Configure a job's reducer implementation.

setDataModelClass

public static void setDataModelClass(JobConf job,
                     Class<? extends GenericData> modelClass)

Configure a job's data model implementation class.

getDataModelClass

public static Class<? extends GenericData> getDataModelClass(Configuration conf)

Return the job's data model implementation class.

createDataModel

public static GenericData createDataModel(Configuration conf)

createInputDataModel

public static GenericData createInputDataModel(Configuration conf)

createMapOutputDataModel

public static GenericData createMapOutputDataModel(Configuration conf)