org.apache.avro.mapred.tether

Class TetherJob

java.lang.Object

org.apache.hadoop.conf.Configured

org.apache.avro.mapred.tether.TetherJob

All Implemented Interfaces:

org.apache.hadoop.conf.Configurable

public class TetherJob
extends org.apache.hadoop.conf.Configured

Constructs and submits tether jobs. This may be used as an API-based method to launch tether jobs.

Field Summary

Fields

Modifier and Type	Field and Description
static String	TETHER_EXEC
static String	TETHER_EXEC_ARGS
static String	TETHER_EXEC_CACHED
static String	TETHER_PROTOCOL

Constructor Summary

Constructors

Constructor and Description
TetherJob()

Method Summary

Modifier and Type	Method and Description
static URI	getExecutable(org.apache.hadoop.mapred.JobConf job) Get the URI of the application's executable.
static TetheredProcess.Protocol	getProtocol(org.apache.hadoop.mapred.JobConf job) Extract from the job configuration file an instance of the TRANSPROTO enumeration to represent the protocol to use for the communication
static org.apache.hadoop.mapred.RunningJob	runJob(org.apache.hadoop.mapred.JobConf job) Submit a job to the map/reduce cluster.
static void	setExecutable(org.apache.hadoop.mapred.JobConf job, File executable) Set the URI for the application's executable.
static void	setExecutable(org.apache.hadoop.mapred.JobConf job, File executable, List<String> args, boolean cached) Set the URI for the application's executable (i.e the program to run in a subprocess and provides the mapper/reducer).
static void	setProtocol(org.apache.hadoop.mapred.JobConf job, String proto) Determines which transport protocol (e.g http or sasl) used to communicate between the parent and subprocess
static org.apache.hadoop.mapred.RunningJob	submitJob(org.apache.hadoop.mapred.JobConf conf) Submit a job to the Map-Reduce framework.

Methods inherited from class org.apache.hadoop.conf.Configured

getConf, setConf

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

TETHER_EXEC

public static final String TETHER_EXEC

See Also:

Constant Field Values

TETHER_EXEC_ARGS

public static final String TETHER_EXEC_ARGS

See Also:

Constant Field Values

TETHER_EXEC_CACHED

public static final String TETHER_EXEC_CACHED

See Also:

Constant Field Values

TETHER_PROTOCOL

public static final String TETHER_PROTOCOL

See Also:

Constant Field Values

Constructor Detail

TetherJob

public TetherJob()

Method Detail

getExecutable

public static URI getExecutable(org.apache.hadoop.mapred.JobConf job)

Get the URI of the application's executable.

setExecutable

public static void setExecutable(org.apache.hadoop.mapred.JobConf job,
                                 File executable)

Set the URI for the application's executable. Normally this in HDFS.

setExecutable

public static void setExecutable(org.apache.hadoop.mapred.JobConf job,
                                 File executable,
                                 List<String> args,
                                 boolean cached)

Set the URI for the application's executable (i.e the program to run in a subprocess and provides the mapper/reducer).

Parameters:

job - - Job

executable - - The URI of the executable

args - - List of additional arguments; Null if no arguments

cached - - If true, the executable URI is cached using DistributedCache - if false its not cached. I.e if the file is already stored on each local file system or if its on a NFS share

getProtocol

public static TetheredProcess.Protocol getProtocol(org.apache.hadoop.mapred.JobConf job)

Extract from the job configuration file an instance of the TRANSPROTO enumeration to represent the protocol to use for the communication

Parameters:

job -

Returns:

runJob

public static org.apache.hadoop.mapred.RunningJob runJob(org.apache.hadoop.mapred.JobConf job)
                                                  throws IOException

Submit a job to the map/reduce cluster. All of the necessary modifications to the job to run under tether are made to the configuration.

Throws:

IOException

submitJob

public static org.apache.hadoop.mapred.RunningJob submitJob(org.apache.hadoop.mapred.JobConf conf)
                                                     throws IOException

Submit a job to the Map-Reduce framework.

Throws:

IOException

setProtocol

public static void setProtocol(org.apache.hadoop.mapred.JobConf job,
                               String proto)
                        throws IOException

Determines which transport protocol (e.g http or sasl) used to communicate between the parent and subprocess

Parameters:

job - - job configuration

proto - - String identifying the protocol currently http or sasl

Throws:

IOException