Namespaces | Classes | Typedefs | Enumerations | Functions

avro Namespace Reference

Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements. More...

Namespaces

namespace  concepts
 

The concept classes are used to simplify NodeImpl.


Classes

struct  is_serializable
struct  is_promotable
struct  type_to_avro
class  CompilerContext
 This class is used to implement an avro spec parser using a flex/bison compiler. More...
class  CompilerNode
 This is a generic "untyped" node that may store values for all possible attributes of Avro complex types. More...
class  DataFileWriterBase
 Type-independent portion of DataFileWriter. More...
class  DataFileWriter
 An Avro datafile that can store objects of type T. More...
class  DataFileReaderBase
class  DataFileReader
class  Decoder
class  ResolvingDecoder
class  Encoder
class  Exception
 Wrapper for std::runtime_error that provides convenience constructor for boost::format objects. More...
class  GenericDatum
class  GenericContainer
class  GenericRecord
class  GenericArray
class  GenericMap
class  GenericEnum
class  GenericFixed
class  GenericReader
class  GenericWriter
struct  codec_traits< std::pair< ValidSchema, GenericDatum > >
class  Layout
class  PrimitiveLayout
class  CompoundLayout
class  Node
 Node is the building block for parse trees. More...
class  NodeImpl
 Implementation details for Node. More...
class  NodePrimitive
class  NodeSymbolic
class  NodeRecord
class  NodeEnum
class  NodeArray
class  NodeMap
class  NodeUnion
class  NodeFixed
class  Parser
 Class that wraps a reader or ValidatingReade with an interface that uses explicit get* names instead of getValue. More...
class  ReaderImpl
 Parses from an avro encoding to the requested type. More...
class  Resolver
class  ResolverSchema
class  ResolvingReader
class  Schema
 The root Schema object is a base class. Nobody constructs this class directly. More...
class  NullSchema
class  BoolSchema
class  IntSchema
class  LongSchema
class  FloatSchema
class  DoubleSchema
class  StringSchema
class  BytesSchema
class  RecordSchema
class  EnumSchema
class  ArraySchema
class  MapSchema
class  UnionSchema
class  FixedSchema
class  Serializer
 Class that wraps a Writer or ValidatingWriter with an interface that uses explicit write* names instead of writeValue. More...
struct  codec_traits
struct  codec_traits< bool >
struct  codec_traits< int32_t >
struct  codec_traits< int64_t >
struct  codec_traits< float >
struct  codec_traits< double >
struct  codec_traits< std::string >
struct  codec_traits< std::vector< uint8_t > >
struct  codec_traits< boost::array< uint8_t, N > >
struct  codec_traits< std::vector< T > >
struct  codec_traits< std::map< std::string, T > >
class  InputStream
class  OutputStream
struct  StreamReader
 A convenience class for reading from an InputStream. More...
struct  StreamWriter
 A convinience class to write data into an OutputStream. More...
class  SymbolMap
 Avro schemas can include types that were previously defined with names in the same avro schema. More...
struct  Null
 define a type to identify Null in template functions More...
class  NullValidator
class  Validator
 This class is used by both the ValidatingSerializer and ValidationParser objects. More...
class  ValidSchema
 A ValidSchema is basically a non-mutable Schema that has passed some minumum of sanity checks. More...
class  WriterImpl
 Class for writing avro data to a stream. More...

Typedefs

typedef boost::array< uint8_t, 16 > DataFileSync
typedef boost::shared_ptr
< Decoder
DecoderPtr
typedef boost::shared_ptr
< ResolvingDecoder
ResolvingDecoderPtr
typedef boost::shared_ptr
< Encoder
EncoderPtr
typedef boost::shared_ptr< NodeNodePtr
typedef concepts::NoAttribute
< std::string > 
NoName
typedef
concepts::SingleAttribute
< std::string > 
HasName
typedef concepts::NoAttribute
< NodePtr > 
NoLeaves
typedef
concepts::SingleAttribute
< NodePtr > 
SingleLeaf
typedef
concepts::MultiAttribute
< NodePtr > 
MultiLeaves
typedef concepts::NoAttribute
< std::string > 
NoLeafNames
typedef
concepts::MultiAttribute
< std::string > 
LeafNames
typedef concepts::NoAttribute
< int > 
NoSize
typedef
concepts::SingleAttribute< int > 
HasSize
typedef NodeImpl< NoName,
NoLeaves, NoLeafNames, NoSize
NodeImplPrimitive
typedef NodeImpl< HasName,
NoLeaves, NoLeafNames, NoSize
NodeImplSymbolic
typedef NodeImpl< HasName,
MultiLeaves, LeafNames, NoSize
NodeImplRecord
typedef NodeImpl< HasName,
NoLeaves, LeafNames, NoSize
NodeImplEnum
typedef NodeImpl< NoName,
SingleLeaf, NoLeafNames,
NoSize
NodeImplArray
typedef NodeImpl< NoName,
MultiLeaves, NoLeafNames,
NoSize
NodeImplMap
typedef NodeImpl< NoName,
MultiLeaves, NoLeafNames,
NoSize
NodeImplUnion
typedef NodeImpl< HasName,
NoLeaves, NoLeafNames, HasSize
NodeImplFixed
typedef ReaderImpl< NullValidatorReader
typedef ReaderImpl< ValidatorValidatingReader
typedef WriterImpl< NullValidatorWriter
typedef WriterImpl< ValidatorValidatingWriter

Enumerations

enum  SchemaResolution {
  RESOLVE_NO_MATCH, RESOLVE_MATCH, RESOLVE_PROMOTABLE_TO_LONG, RESOLVE_PROMOTABLE_TO_FLOAT,
  RESOLVE_PROMOTABLE_TO_DOUBLE
}
enum  Type {
  AVRO_STRING, AVRO_BYTES, AVRO_INT, AVRO_LONG,
  AVRO_FLOAT, AVRO_DOUBLE, AVRO_BOOL, AVRO_NULL,
  AVRO_RECORD, AVRO_ENUM, AVRO_ARRAY, AVRO_MAP,
  AVRO_UNION, AVRO_FIXED, AVRO_NUM_TYPES, AVRO_SYMBOLIC = AVRO_NUM_TYPES,
  AVRO_UNKNOWN = -1
}

Functions

template<typename Reader , typename T >
void parse (Reader &p, T &val)
 The main parse entry point function.
template<typename T >
void parse (ResolvingReader &p, T &val)
template<typename Reader , typename T >
void translatingParse (Reader &p, T &val, const boost::false_type &)
template<typename Writer , typename T >
void serialize (Writer &s, const T &val)
 The main serializer entry point function.
template<typename Writer , typename T >
void serialize (Writer &s, const T &val, const boost::false_type &)
 Type trait should be set to is_serializable in otherwise force the compiler to complain.
void compileJsonSchema (std::istream &is, ValidSchema &schema)
 Given a stream comtaining a JSON schema, compiles the schema to a ValidSchema object.
bool compileJsonSchema (std::istream &is, ValidSchema &schema, std::string &error)
 Non-throwing version of compileJsonSchema.
NodePtr nodeFromCompilerNode (CompilerNode &compilerNode)
DecoderPtr binaryDecoder ()
 Returns an decoder that can decode binary Avro standard.
DecoderPtr validatingDecoder (const ValidSchema &schema, const DecoderPtr &base)
 Returns an decoder that validates sequence of calls to an underlying Decoder against the given schema.
DecoderPtr jsonDecoder (const ValidSchema &schema)
 Returns an decoder that can decode Avro standard for JSON.
ResolvingDecoderPtr resolvingDecoder (const ValidSchema &writer, const ValidSchema &reader, const DecoderPtr &base)
 Returns a decoder that decodes avro data from base written according to writerSchema and resolves against readerSchema.
EncoderPtr binaryEncoder ()
 Returns an encoder that can encode binary Avro standard.
EncoderPtr validatingEncoder (const ValidSchema &schema, const EncoderPtr &base)
 Returns an encoder that validates sequence of calls to an underlying Encoder against the given schema.
EncoderPtr jsonEncoder (const ValidSchema &schema)
 Returns an encoder that can encode Avro standard for JSON.
NodePtr resolveSymbol (const NodePtr &node)
Type nextType (Parser< ValidatingReader > &p)
bool currentRecordName (Parser< ValidatingReader > &p, std::string &name)
bool nextFieldName (Parser< ValidatingReader > &p, std::string &name)
ResolverconstructResolver (const ValidSchema &rwriterSchema, const ValidSchema &readerSchema, const Layout &readerLayout)
template<typename T >
void encode (Encoder &e, const T &t)
template<typename T >
void decode (Decoder &d, T &t)
std::auto_ptr< OutputStreammemoryOutputStream (size_t chunkSize=4 *1024)
 Returns a new OutputStream, which grows in memory chunks of specified size.
std::auto_ptr< InputStreammemoryInputStream (const uint8_t *data, size_t len)
 Returns a new InputStream, with the data from the given byte array.
std::auto_ptr< InputStreammemoryInputStream (const OutputStream &source)
 Returns a new InputStream with the contents written into an outputstream.
std::auto_ptr< OutputStreamfileOutputStream (const char *filename, size_t bufferSize=8 *1024)
 Returns a new OutputStream whose contents would be stored in a file.
std::auto_ptr< InputStreamfileInputStream (const char *filename, size_t bufferSize=8 *1024)
 Returns a new InputStream whose contents come from the given file.
void copy (InputStream &in, OutputStream &out)
 A convenience function to copy all the contents of an input stream into an output stream.
bool isPrimitive (Type t)
bool isCompound (Type t)
bool isAvroType (Type t)
bool isAvroTypeOrPseudoType (Type t)
const std::string & toString (Type type)
std::ostream & operator<< (std::ostream &os, avro::Type type)
std::ostream & operator<< (std::ostream &os, const Null &null)
uint64_t encodeZigzag64 (int64_t input)
int64_t decodeZigzag64 (uint64_t input)
uint32_t encodeZigzag32 (int32_t input)
int32_t decodeZigzag32 (uint32_t input)
size_t encodeInt32 (int32_t input, boost::array< uint8_t, 5 > &output)
size_t encodeInt64 (int64_t input, boost::array< uint8_t, 10 > &output)

template<typename Reader , typename T >
void parse (Reader &p, T &val, const boost::false_type &)
 Type trait should be set to is_serializable in otherwise force the compiler to complain.
template<typename Reader >
void parse (Reader &p, std::vector< uint8_t > &val, const boost::true_type &)
template<typename T >
void translatingParse (ResolvingReader &p, T &val, const boost::true_type &)

template<typename Writer , typename T >
void serialize (Writer &s, T val, const boost::true_type &)
 The remainder of the file includes default implementations for serializable types.
template<typename Writer >
void serialize (Writer &s, const std::vector< uint8_t > &val, const boost::true_type &)

Detailed Description

Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements.

See the NOTICE file distributed with this work for additional information regarding copyright ownership. The ASF licenses this file to you under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

See the NOTICE file distributed with this work for additional information regarding copyright ownership. The ASF licenses this file to you under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. A bunch of templates and specializations for encoding and decoding specific types.

Primitive AVRO types BOOLEAN, INT, LONG, FLOAT, DOUBLE, STRING and BYTES get decoded to and encoded from C++ types bool, int32_t, int64_t, float, double, std::string and std::vector<uint8_t> respectively. In addition, std::vector<T> for aribtrary type T gets encoded as an Avro array of T. Similarly, std::map<std::string, T> for arbitrary type T gets encoded as an Avro map with value type T.

Users can have their custom types encoded/decoded by specializing avro::codec_traits class for their types.


Enumeration Type Documentation

Enumerator:
RESOLVE_NO_MATCH 

The schemas definitely do not match.

RESOLVE_MATCH 

The schemas match at a cursory level.

For records and enums, this means the name is the same, but it does not necessarily mean that every symbol or field is an exact match.

RESOLVE_PROMOTABLE_TO_LONG 

For primitives, the matching may occur if the type is promotable.

This means that the writer matches reader if the writer's type is promoted the specified type.


Function Documentation

void avro::compileJsonSchema ( std::istream &  is,
ValidSchema &  schema 
)

Given a stream comtaining a JSON schema, compiles the schema to a ValidSchema object.

Throws if the schema cannot be compiled to a valid schema

bool avro::compileJsonSchema ( std::istream &  is,
ValidSchema &  schema,
std::string &  error 
)

Non-throwing version of compileJsonSchema.

Returns:
True if no error, false if error (with the error string set)
std::auto_ptr<InputStream> avro::fileInputStream ( const char *  filename,
size_t  bufferSize = 8 *1024 
)

Returns a new InputStream whose contents come from the given file.

Data is read in chunks of given buffer size.

std::auto_ptr<OutputStream> avro::fileOutputStream ( const char *  filename,
size_t  bufferSize = 8 *1024 
)

Returns a new OutputStream whose contents would be stored in a file.

Data is written in chunks of given buffer size.

If there is a file with the given name, it is truncated and overwritten. If there is no file with the given name, it is created.

std::auto_ptr<InputStream> avro::memoryInputStream ( const uint8_t *  data,
size_t  len 
)

Returns a new InputStream, with the data from the given byte array.

It does not copy the data, the byte array should remain valid until the InputStream is used.

std::auto_ptr<InputStream> avro::memoryInputStream ( const OutputStream &  source  ) 

Returns a new InputStream with the contents written into an outputstream.

The output stream must have been returned by an earlier call to memoryOutputStream(). The contents for the new input stream are the snapshot of the outputstream. One can construct any number of memory input stream from a single memory output stream.

template<typename Reader , typename T >
void avro::parse ( Reader &  p,
T &  val,
const boost::false_type  
)

Type trait should be set to is_serializable in otherwise force the compiler to complain.

The remainder of the file includes default implementations for serializable types.

template<typename Reader , typename T >
void avro::parse ( Reader &  p,
T &  val 
)

The main parse entry point function.

Takes a parser (either validating or plain) and the object that should receive the parsed data.

ResolvingDecoderPtr avro::resolvingDecoder ( const ValidSchema &  writer,
const ValidSchema &  reader,
const DecoderPtr &  base 
)

Returns a decoder that decodes avro data from base written according to writerSchema and resolves against readerSchema.

The client uses the decoder as if the data were written using readerSchema. // FIXME: Handle out of order fields.

template<typename Writer , typename T >
void avro::serialize ( Writer &  s,
const T &  val 
)

The main serializer entry point function.

Takes a serializer (either validating or plain) and the object that should be serialized.