Package org.apache.avro.mapred

Class AvroJob

java.lang.Object
org.apache.avro.mapred.AvroJob
public class AvroJob extends Object
Setters to configure jobs for Avro data.

  • Field Details

    • INPUT_SCHEMA

      public static final String INPUT_SCHEMA
      The configuration key for a job's input schema.
      See Also:

    • MAP_OUTPUT_SCHEMA

      public static final String MAP_OUTPUT_SCHEMA
      The configuration key for a job's intermediate schema.
      See Also:

    • OUTPUT_SCHEMA

      public static final String OUTPUT_SCHEMA
      The configuration key for a job's output schema.
      See Also:

    • OUTPUT_CODEC

      public static final String OUTPUT_CODEC
      The configuration key for a job's output compression codec. This takes one of the strings registered in CodecFactory
      See Also:

    • TEXT_PREFIX

      public static final String TEXT_PREFIX
      The configuration key prefix for a text output metadata.
      See Also:

    • BINARY_PREFIX

      public static final String BINARY_PREFIX
      The configuration key prefix for a binary output metadata.
      See Also:

    • INPUT_IS_REFLECT

      public static final String INPUT_IS_REFLECT
      The configuration key for reflection-based input representation.
      See Also:

    • MAP_OUTPUT_IS_REFLECT

      public static final String MAP_OUTPUT_IS_REFLECT
      The configuration key for reflection-based map output representation.
      See Also:

  • Method Details

    • setInputSchema

      public static void setInputSchema(JobConf job, Schema s)
      Configure a job's map input schema.

    • getInputSchema

      public static Schema getInputSchema(Configuration job)
      Return a job's map input schema.

    • setMapOutputSchema

      public static void setMapOutputSchema(JobConf job, Schema s)
      Configure a job's map output schema. The map output schema defaults to the output schema and need only be specified when it differs. Thus must be a Pair schema.

    • getMapOutputSchema

      public static Schema getMapOutputSchema(Configuration job)
      Return a job's map output key schema.

    • setOutputSchema

      public static void setOutputSchema(JobConf job, Schema s)
      Configure a job's output schema. Unless this is a map-only job, this must be a Pair schema.

    • setOutputCodec

      public static void setOutputCodec(JobConf job, String codec)
      Configure a job's output compression codec.

    • setOutputMeta

      public static void setOutputMeta(JobConf job, String key, String value)
      Add metadata to job output files.

    • setOutputMeta

      public static void setOutputMeta(JobConf job, String key, long value)
      Add metadata to job output files.

    • setOutputMeta

      public static void setOutputMeta(JobConf job, String key, byte[] value)
      Add metadata to job output files.

    • setInputSequenceFile

      public static void setInputSequenceFile(JobConf job)
      Indicate that a job's input files are in SequenceFile format.

    • setReflect

      public static void setReflect(JobConf job)
      Indicate that all a job's data should use the reflect representation.

    • setInputReflect

      public static void setInputReflect(JobConf job)
      Indicate that a job's input data should use reflect representation.

    • setMapOutputReflect

      public static void setMapOutputReflect(JobConf job)
      Indicate that a job's map output data should use reflect representation.

    • getOutputSchema

      public static Schema getOutputSchema(Configuration job)
      Return a job's output key schema.

    • setMapperClass

      public static void setMapperClass(JobConf job, Class<? extends AvroMapper> c)
      Configure a job's mapper implementation.

    • setCombinerClass

      public static void setCombinerClass(JobConf job, Class<? extends AvroReducer> c)
      Configure a job's combiner implementation.

    • setReducerClass

      public static void setReducerClass(JobConf job, Class<? extends AvroReducer> c)
      Configure a job's reducer implementation.

    • setDataModelClass

      public static void setDataModelClass(JobConf job, Class<? extends GenericData> modelClass)
      Configure a job's data model implementation class.

    • getDataModelClass

      public static Class<? extends GenericData> getDataModelClass(Configuration conf)
      Return the job's data model implementation class.

    • createDataModel

      public static GenericData createDataModel(Configuration conf)

    • createInputDataModel

      public static GenericData createInputDataModel(Configuration conf)

    • createMapOutputDataModel

      public static GenericData createMapOutputDataModel(Configuration conf)