Package org.apache.trevni.avro.mapreduce
Class AvroTrevniKeyValueInputFormat<K,V>
java.lang.Object
org.apache.hadoop.mapreduce.InputFormat<AvroKey<K>,AvroValue<V>>
org.apache.hadoop.mapreduce.lib.input.FileInputFormat<AvroKey<K>,AvroValue<V>>
org.apache.trevni.avro.mapreduce.AvroTrevniKeyValueInputFormat<K,V>
- Type Parameters:
K
- The type of the Trevni key to read.V
- The type of the Trevni value to read.A subset schema to be read may be specified with
AvroJob.setInputKeySchema(org.apache.hadoop.mapreduce.Job, org.apache.avro.Schema)
andAvroJob.setInputValueSchema(org.apache.hadoop.mapreduce.Job, org.apache.avro.Schema)
.
An
InputFormat
for Trevni files.
This implement was modeled off
AvroKeyValueInputFormat
to allow for easy
transition
A MapReduce InputFormat that reads from Trevni container files of key/value generic records.
Trevni container files that container generic records with the two fields 'key' and 'value' are expected. The contents of the 'key' field will be used as the job input key, and the contents of the 'value' field will be used as the job output value.
-
Nested Class Summary
Nested classes/interfaces inherited from class org.apache.hadoop.mapreduce.lib.input.FileInputFormat
FileInputFormat.Counter
-
Field Summary
Fields inherited from class org.apache.hadoop.mapreduce.lib.input.FileInputFormat
DEFAULT_LIST_STATUS_NUM_THREADS, INPUT_DIR, INPUT_DIR_NONRECURSIVE_IGNORE_SUBDIRS, INPUT_DIR_RECURSIVE, LIST_STATUS_NUM_THREADS, NUM_INPUT_FILES, PATHFILTER_CLASS, SPLIT_MAXSIZE, SPLIT_MINSIZE
-
Constructor Summary
-
Method Summary
Modifier and TypeMethodDescriptioncreateRecordReader
(InputSplit split, TaskAttemptContext context) Methods inherited from class org.apache.hadoop.mapreduce.lib.input.FileInputFormat
addInputPath, addInputPathRecursively, addInputPaths, computeSplitSize, getBlockIndex, getFormatMinSplitSize, getInputDirRecursive, getInputPathFilter, getInputPaths, getMaxSplitSize, getMinSplitSize, getSplits, isSplitable, listStatus, makeSplit, makeSplit, setInputDirRecursive, setInputPathFilter, setInputPaths, setInputPaths, setMaxInputSplitSize, setMinInputSplitSize
-
Constructor Details
-
AvroTrevniKeyValueInputFormat
public AvroTrevniKeyValueInputFormat()
-
-
Method Details
-
createRecordReader
public RecordReader<AvroKey<K>,AvroValue<V>> createRecordReader(InputSplit split, TaskAttemptContext context) throws IOException, InterruptedException - Specified by:
createRecordReader
in classInputFormat<AvroKey<K>,
AvroValue<V>> - Throws:
IOException
InterruptedException
-