A bunch of templates and specializations for encoding and decoding specific types. More...

Namespaces
	concepts
	The concept classes are used to simplify NodeImpl.

Classes
class	ArraySchema

class	BoolSchema

class	BytesSchema

struct	codec_traits
	Codec_traits tells avro how to encode and decode an object of given type. More...

struct	codec_traits< avro::null >
	codec_traits for Avro null. More...

struct	codec_traits< bool >
	codec_traits for Avro boolean. More...

struct	codec_traits< double >
	codec_traits for Avro double. More...

struct	codec_traits< float >
	codec_traits for Avro float. More...

struct	codec_traits< GenericDatum >
	Specialization of codec_traits for GenericDatum. More...

struct	codec_traits< int32_t >
	codec_traits for Avro int. More...

struct	codec_traits< int64_t >
	codec_traits for Avro long. More...

struct	codec_traits< std::array< uint8_t, N > >
	codec_traits for Avro fixed. More...

struct	codec_traits< std::conditional< avro::is_not_defined< bool_codec_traits >::value, std::vector< bool >::const_reference, void >::type >

struct	codec_traits< std::map< std::string, T > >
	codec_traits for Avro maps. More...

struct	codec_traits< std::pair< ValidSchema, GenericDatum > >
	Specialization of codec_traits for Generic datum along with its schema. More...

struct	codec_traits< std::string >
	codec_traits for Avro string. More...

struct	codec_traits< std::vector< T > >
	codec_traits for Avro arrays. More...

struct	codec_traits< std::vector< uint8_t > >
	codec_traits for Avro bytes. More...

class	CompoundLayout

class	CustomFields

class	DataFileReader
	Reads the contents of data file one after another. More...

class	DataFileReaderBase
	The type independent portion of reader. More...

class	DataFileWriter
	An Avro datafile that can store objects of type T. More...

class	DataFileWriterBase
	Type-independent portion of DataFileWriter. More...

class	Decoder
	Decoder is an interface implemented by every decoder capable of decoding Avro data. More...

class	DoubleSchema

class	Encoder
	The abstract base class for all Avro encoders. More...

class	EnumSchema

class	Exception
	Wrapper for std::runtime_error that provides convenience constructor for boost::format objects. More...

class	FixedSchema

class	FloatSchema

class	GenericArray
	The generic container for Avro arrays. More...

class	GenericContainer
	The base class for all generic type for containers. More...

class	GenericDatum
	Generic datum which can hold any Avro type. More...

class	GenericEnum
	Generic container for Avro enum. More...

class	GenericFixed
	Generic container for Avro fixed. More...

class	GenericMap
	The generic container for Avro maps. More...

class	GenericReader
	A utility class to read generic datum from decoders. More...

class	GenericRecord
	The generic container for Avro records. More...

class	GenericUnion
	Generic container for unions. More...

class	GenericWriter
	A utility class to write generic datum to encoders. More...

class	InputStream
	A no-copy input stream. More...

class	IntSchema

struct	is_defined
	Check if a `T` is a complete type i.e. More...

struct	is_not_defined
	Similar to is_defined, but used to check if T is not defined. More...

struct	is_promotable

struct	is_serializable
	Define an is_serializable trait for types we can serialize natively. More...

class	Layout

class	LogicalType

class	LongSchema

class	MapSchema

class	Name

class	Node
	Node is the building block for parse trees. More...

class	NodeArray

class	NodeEnum

class	NodeFixed

class	NodeImpl
	Implementation details for Node. More...

class	NodeMap

class	NodePrimitive

class	NodeRecord

class	NodeSymbolic

class	NodeUnion

struct	Null
	define a type to represent Avro Null in template functions More...

class	NullSchema

class	NullValidator

class	OutputStream
	A no-copy output stream. More...

class	Parser
	Class that wraps a reader or ValidatingReade with an interface that uses explicit get* names instead of getValue. More...

class	PrimitiveLayout

class	ReaderImpl
	Parses from an avro encoding to the requested type. More...

class	RecordSchema

class	Resolver

class	ResolverSchema

class	ResolvingDecoder
	ResolvingDecoder is derived from Decoder, with an additional function to obtain the field ordering of fields within a record. More...

class	ResolvingReader

class	Schema
	The root Schema object is a base class. Nobody constructs this class directly. More...

class	SeekableInputStream
	An InputStream which also supports seeking to a specific offset. More...

class	Serializer
	Class that wraps a Writer or ValidatingWriter with an interface that uses explicit write* names instead of writeValue. More...

struct	StreamReader
	A convenience class for reading from an InputStream. More...

struct	StreamWriter
	A convenience class to write data into an OutputStream. More...

class	StringSchema

class	SymbolicSchema

struct	type_to_avro

class	UnionSchema

class	Validator
	This class is used by both the ValidatingSerializer and ValidationParser objects. More...

class	ValidSchema
	A ValidSchema is basically a non-mutable Schema that has passed some minimum of sanity checks. More...

class	WriterImpl
	Class for writing avro data to a stream. More...

Typedefs
typedef std::array< uint8_t, SyncSize >	DataFileSync
	The sync value.

using	DecoderPtr = std::shared_ptr< Decoder >
	Shared pointer to Decoder.

using	ResolvingDecoderPtr = std::shared_ptr< ResolvingDecoder >
	Shared pointer to ResolvingDecoder.

using	EncoderPtr = std::shared_ptr< Encoder >
	Shared pointer to Encoder.

using	NodePtr = std::shared_ptr< Node >

using	NoName = concepts::NoAttribute< Name >

using	HasName = concepts::SingleAttribute< Name >

using	HasDoc = concepts::SingleAttribute< std::string >

using	NoLeaves = concepts::NoAttribute< NodePtr >

using	SingleLeaf = concepts::SingleAttribute< NodePtr >

using	MultiLeaves = concepts::MultiAttribute< NodePtr >

using	NoLeafNames = concepts::NoAttribute< std::string >

using	LeafNames = concepts::MultiAttribute< std::string >

using	MultiAttributes = concepts::MultiAttribute< CustomFields >

using	NoAttributes = concepts::NoAttribute< CustomFields >

using	NoSize = concepts::NoAttribute< int >

using	HasSize = concepts::SingleAttribute< int >

using	NodeImplPrimitive = NodeImpl< NoName, NoLeaves, NoLeafNames, MultiAttributes, NoSize >

using	NodeImplSymbolic = NodeImpl< HasName, NoLeaves, NoLeafNames, NoAttributes, NoSize >

using	NodeImplRecord = NodeImpl< HasName, MultiLeaves, LeafNames, MultiAttributes, NoSize >

using	NodeImplEnum = NodeImpl< HasName, NoLeaves, LeafNames, NoAttributes, NoSize >

using	NodeImplArray = NodeImpl< NoName, SingleLeaf, NoLeafNames, NoAttributes, NoSize >

using	NodeImplMap = NodeImpl< NoName, MultiLeaves, NoLeafNames, NoAttributes, NoSize >

using	NodeImplUnion = NodeImpl< NoName, MultiLeaves, NoLeafNames, NoAttributes, NoSize >

using	NodeImplFixed = NodeImpl< HasName, NoLeaves, NoLeafNames, NoAttributes, HasSize >

using	Reader = ReaderImpl< NullValidator >

using	ValidatingReader = ReaderImpl< Validator >

typedef boost::blank	null

typedef codec_traits< std::vector< bool >::const_reference >	bool_codec_traits

typedef std::unique_ptr< InputStream >	InputStreamPtr

typedef std::unique_ptr< SeekableInputStream >	SeekableInputStreamPtr

typedef std::unique_ptr< OutputStream >	OutputStreamPtr

using	Writer = WriterImpl< NullValidator >

using	ValidatingWriter = WriterImpl< Validator >

Enumerations
enum	Codec { NULL_CODEC, DEFLATE_CODEC }
	Specify type of compression to use when writing data files.

enum	SchemaResolution { RESOLVE_NO_MATCH, RESOLVE_MATCH, RESOLVE_PROMOTABLE_TO_LONG, RESOLVE_PROMOTABLE_TO_FLOAT, RESOLVE_PROMOTABLE_TO_DOUBLE }

enum	Type { AVRO_STRING, AVRO_BYTES, AVRO_INT, AVRO_LONG, AVRO_FLOAT, AVRO_DOUBLE, AVRO_BOOL, AVRO_NULL, AVRO_RECORD, AVRO_ENUM, AVRO_ARRAY, AVRO_MAP, AVRO_UNION, AVRO_FIXED, AVRO_NUM_TYPES, AVRO_SYMBOLIC = AVRO_NUM_TYPES, AVRO_UNKNOWN = -1 }
	The "type" for the schema. More...

Functions
template<typename Reader , typename T >
void	parse (Reader &p, T &val)
	The main parse entry point function. More...

template<typename T >
void	parse (ResolvingReader &p, T &val)

template<typename Reader , typename T >
void	parse (Reader &p, T &val, const std::false_type &)
	Type trait should be set to is_serializable in otherwise force the compiler to complain.

template<typename Reader , typename T >
void	translatingParse (Reader &p, T &val, const std::false_type &)

template<typename Reader , typename T >
void	parse (Reader &p, T &val, const std::true_type &)
	The remainder of the file includes default implementations for serializable types.

template<typename Reader >
void	parse (Reader &p, std::vector< uint8_t > &val, const std::true_type &)

template<typename T >
void	translatingParse (ResolvingReader &p, T &val, const std::true_type &)

template<typename Writer , typename T >
void	serialize (Writer &s, const T &val)
	The main serializer entry point function. More...

template<typename Writer , typename T >
void	serialize (Writer &s, const T &val, const std::false_type &)
	Type trait should be set to is_serializable in otherwise force the compiler to complain.

template<typename Writer , typename T >
void	serialize (Writer &s, T val, const std::true_type &)
	The remainder of the file includes default implementations for serializable types.

template<typename Writer >
void	serialize (Writer &s, const std::vector< uint8_t > &val, const std::true_type &)

AVRO_DECL void	compileJsonSchema (std::istream &is, ValidSchema &schema)
	Given a stream containing a JSON schema, compiles the schema to a ValidSchema object. More...

AVRO_DECL bool	compileJsonSchema (std::istream &is, ValidSchema &schema, std::string &error)
	Non-throwing version of compileJsonSchema. More...

AVRO_DECL ValidSchema	compileJsonSchemaFromStream (InputStream &is)

AVRO_DECL ValidSchema	compileJsonSchemaFromMemory (const uint8_t *input, size_t len)

AVRO_DECL ValidSchema	compileJsonSchemaFromString (const char *input)

AVRO_DECL ValidSchema	compileJsonSchemaFromString (const std::string &input)

AVRO_DECL ValidSchema	compileJsonSchemaFromFile (const char *filename)

AVRO_DECL DecoderPtr	binaryDecoder ()
	Returns an decoder that can decode binary Avro standard.

AVRO_DECL DecoderPtr	validatingDecoder (const ValidSchema &schema, const DecoderPtr &base)
	Returns an decoder that validates sequence of calls to an underlying Decoder against the given schema.

AVRO_DECL DecoderPtr	jsonDecoder (const ValidSchema &schema)
	Returns an decoder that can decode Avro standard for JSON.

AVRO_DECL ResolvingDecoderPtr	resolvingDecoder (const ValidSchema &writer, const ValidSchema &reader, const DecoderPtr &base)
	Returns a decoder that decodes avro data from base written according to writerSchema and resolves against readerSchema. More...

AVRO_DECL EncoderPtr	binaryEncoder ()
	Returns an encoder that can encode binary Avro standard.

AVRO_DECL EncoderPtr	validatingEncoder (const ValidSchema &schema, const EncoderPtr &base)
	Returns an encoder that validates sequence of calls to an underlying Encoder against the given schema.

AVRO_DECL EncoderPtr	jsonEncoder (const ValidSchema &schema)
	Returns an encoder that encodes Avro standard for JSON.

AVRO_DECL EncoderPtr	jsonPrettyEncoder (const ValidSchema &schema)
	Returns an encoder that encodes Avro standard for pretty printed JSON.

std::ostream &	operator<< (std::ostream &os, const Name &n)

NodePtr	resolveSymbol (const NodePtr &node)

template<typename T >
std::string	intToHex (T i)

Type	nextType (Parser< ValidatingReader > &p)

bool	currentRecordName (Parser< ValidatingReader > &p, std::string &name)

bool	nextFieldName (Parser< ValidatingReader > &p, std::string &name)

std::unique_ptr< Resolver >	constructResolver (const ValidSchema &writerSchema, const ValidSchema &readerSchema, const Layout &readerLayout)

template<typename T >
void	encode (Encoder &e, const T &t)
	Generic encoder function that makes use of the codec_traits.

template<typename T >
void	decode (Decoder &d, T &t)
	Generic decoder function that makes use of the codec_traits.

AVRO_DECL OutputStreamPtr	memoryOutputStream (size_t chunkSize=4 *1024)
	Returns a new OutputStream, which grows in memory chunks of specified size.

AVRO_DECL InputStreamPtr	memoryInputStream (const uint8_t *data, size_t len)
	Returns a new InputStream, with the data from the given byte array. More...

AVRO_DECL InputStreamPtr	memoryInputStream (const OutputStream &source)
	Returns a new InputStream with the contents written into an OutputStream. More...

AVRO_DECL std::shared_ptr< std::vector< uint8_t > >	snapshot (const OutputStream &source)
	Returns the contents written so far into the output stream, which should be a memory output stream. More...

AVRO_DECL OutputStreamPtr	fileOutputStream (const char filename, size_t bufferSize=8 1024)
	Returns a new OutputStream whose contents would be stored in a file. More...

AVRO_DECL InputStreamPtr	fileInputStream (const char filename, size_t bufferSize=8 1024)
	Returns a new InputStream whose contents come from the given file. More...

AVRO_DECL SeekableInputStreamPtr	fileSeekableInputStream (const char filename, size_t bufferSize=8 1024)

AVRO_DECL OutputStreamPtr	ostreamOutputStream (std::ostream &os, size_t bufferSize=8 *1024)
	Returns a new OutputStream whose contents will be sent to the given std::ostream. More...

AVRO_DECL InputStreamPtr	istreamInputStream (std::istream &in, size_t bufferSize=8 *1024)
	Returns a new InputStream whose contents come from the given std::istream. More...

AVRO_DECL InputStreamPtr	nonSeekableIstreamInputStream (std::istream &is, size_t bufferSize=8 *1024)
	Returns a new InputStream whose contents come from the given std::istream. More...

void	copy (InputStream &in, OutputStream &out)
	A convenience function to copy all the contents of an input stream into an output stream.

constexpr bool	isPrimitive (Type t) noexcept
	Returns true if and only if the given type is a primitive. More...

constexpr bool	isCompound (Type t) noexcept
	Returns true if and only if the given type is a non primitive valid type. More...

constexpr bool	isAvroType (Type t) noexcept
	Returns true if and only if the given type is a valid avro type.

constexpr bool	isAvroTypeOrPseudoType (Type t) noexcept
	Returns true if and only if the given type is within the valid range of enumeration.

const AVRO_DECL std::string &	toString (Type type) noexcept
	Converts the given type into a string. More...

AVRO_DECL std::ostream &	operator<< (std::ostream &os, avro::Type type)
	Writes a string form of the given type into the given ostream.

std::ostream &	operator<< (std::ostream &os, const Null &null)
	Writes schema for null `null` type to `os`. More...

constexpr AVRO_DECL uint64_t	encodeZigzag64 (int64_t input) noexcept

constexpr AVRO_DECL int64_t	decodeZigzag64 (uint64_t input) noexcept

constexpr AVRO_DECL uint32_t	encodeZigzag32 (int32_t input) noexcept

constexpr AVRO_DECL int32_t	decodeZigzag32 (uint32_t input) noexcept

AVRO_DECL size_t	encodeInt32 (int32_t input, std::array< uint8_t, 5 > &output) noexcept

AVRO_DECL size_t	encodeInt64 (int64_t input, std::array< uint8_t, 10 > &output) noexcept

Variables
class AVRO_DECL	InputStream

class AVRO_DECL	ValidSchema
	This class is used to implement an avro spec parser using a flex/bison compiler. More...

const int	SyncSize = 16

class AVRO_DECL	Schema

Detailed Description

A bunch of templates and specializations for encoding and decoding specific types.

Primitive AVRO types BOOLEAN, INT, LONG, FLOAT, DOUBLE, STRING and BYTES get decoded to and encoded from C++ types bool, int32_t, int64_t, float, double, std::string and std::vector<uint8_t> respectively. In addition, std::vector<T> for arbitrary type T gets encoded as an Avro array of T. Similarly, std::map<std::string, T> for arbitrary type T gets encoded as an Avro map with value type T.

Users can have their custom types encoded/decoded by specializing avro::codec_traits class for their types.

Enumeration Type Documentation

◆ SchemaResolution

enum avro::SchemaResolution

Enumerator

RESOLVE_NO_MATCH

The schemas definitely do not match.

RESOLVE_MATCH

The schemas match at a cursory level.

For records and enums, this means the name is the same, but it does not
necessarily mean that every symbol or field is an exact match.

RESOLVE_PROMOTABLE_TO_LONG

For primitives, the matching may occur if the type is promotable.

This means that the writer matches reader if the writer's type is promoted the specified type.

◆ Type

enum avro::Type

The "type" for the schema.

Enumerator
AVRO_STRING	String
AVRO_BYTES	Sequence of variable length bytes data
AVRO_INT	32-bit integer
AVRO_LONG	64-bit integer
AVRO_FLOAT	Floating point number
AVRO_DOUBLE	Double precision floating point number
AVRO_BOOL	Boolean value
AVRO_NULL	Null
AVRO_RECORD	Record, a sequence of fields
AVRO_ENUM	Enumeration
AVRO_ARRAY	Homogeneous array of some specific type
AVRO_MAP	Homogeneous map from string to some specific type
AVRO_UNION	Union of one or more types
AVRO_FIXED	Fixed number of bytes
AVRO_NUM_TYPES	Marker
AVRO_SYMBOLIC	User internally to avoid circular references.
AVRO_UNKNOWN	Used internally.

Function Documentation

◆ compileJsonSchema() [1/2]

AVRO_DECL void avro::compileJsonSchema	(	std::istream &	is,
		ValidSchema &	schema
	)

Given a stream containing a JSON schema, compiles the schema to a ValidSchema object.

Throws if the schema cannot be compiled to a valid schema

◆ compileJsonSchema() [2/2]

AVRO_DECL bool avro::compileJsonSchema	(	std::istream &	is,
		ValidSchema &	schema,
		std::string &	error
	)

Non-throwing version of compileJsonSchema.

Returns: True if no error, false if error (with the error string set)

◆ fileInputStream()

AVRO_DECL InputStreamPtr avro::fileInputStream	(	const char *	filename,
		size_t	bufferSize = `8 *1024`
	)

Returns a new InputStream whose contents come from the given file.

Data is read in chunks of given buffer size.

◆ fileOutputStream()

AVRO_DECL OutputStreamPtr avro::fileOutputStream	(	const char *	filename,
		size_t	bufferSize = `8 *1024`
	)

Returns a new OutputStream whose contents would be stored in a file.

Data is written in chunks of given buffer size.

If there is a file with the given name, it is truncated and overwritten. If there is no file with the given name, it is created.

◆ isCompound()

constexpr bool avro::isCompound ( Type t )

inlineconstexprnoexcept

Returns true if and only if the given type is a non primitive valid type.

Primitive types are: string, bytes, int, long, float, double, boolean and null

References AVRO_NUM_TYPES, and AVRO_RECORD.

◆ isPrimitive()

constexpr bool avro::isPrimitive ( Type t )

inlineconstexprnoexcept

Returns true if and only if the given type is a primitive.

Primitive types are: string, bytes, int, long, float, double, boolean and null

References AVRO_RECORD, and AVRO_STRING.

◆ istreamInputStream()

AVRO_DECL InputStreamPtr avro::istreamInputStream	(	std::istream &	in,
		size_t	bufferSize = `8 *1024`
	)

Returns a new InputStream whose contents come from the given std::istream.

The std::istream object should outlive the returned InputStream.

◆ memoryInputStream() [1/2]

AVRO_DECL InputStreamPtr avro::memoryInputStream ( const OutputStream & source )

Returns a new InputStream with the contents written into an OutputStream.

The output stream must have been returned by an earlier call to memoryOutputStream(). The contents for the new InputStream are the snapshot of the output stream. One can construct any number of memory input stream from a single memory output stream.

◆ memoryInputStream() [2/2]

AVRO_DECL InputStreamPtr avro::memoryInputStream	(	const uint8_t *	data,
		size_t	len
	)

Returns a new InputStream, with the data from the given byte array.

It does not copy the data, the byte array should remain valid until the InputStream is used.

◆ nonSeekableIstreamInputStream()

AVRO_DECL InputStreamPtr avro::nonSeekableIstreamInputStream	(	std::istream &	is,
		size_t	bufferSize = `8 *1024`
	)

Returns a new InputStream whose contents come from the given std::istream.

Use this instead of istreamInputStream if the istream does not support seekg (e.g. compressed streams). The returned InputStream would read off bytes instead of seeking. Of, course it has a performance penalty when reading instead of seeking; So, use this only when seekg does not work. The std::istream object should outlive the returned InputStream.

◆ operator<<()

std::ostream& avro::operator<<	(	std::ostream &	os,
		const Null &	null
	)

Writes schema for null null type to os.

Parameters

os	The ostream to write to.
null	The value to be written.

◆ ostreamOutputStream()

AVRO_DECL OutputStreamPtr avro::ostreamOutputStream	(	std::ostream &	os,
		size_t	bufferSize = `8 *1024`
	)

Returns a new OutputStream whose contents will be sent to the given std::ostream.

The std::ostream object should outlive the returned OutputStream.

◆ parse()

template<typename Reader , typename T >

void avro::parse	(	Reader &	p,
		T &	val
	)

The main parse entry point function.

Takes a parser (either validating or plain) and the object that should receive the parsed data.

◆ resolvingDecoder()

AVRO_DECL ResolvingDecoderPtr avro::resolvingDecoder	(	const ValidSchema &	writer,
		const ValidSchema &	reader,
		const DecoderPtr &	base
	)

Returns a decoder that decodes avro data from base written according to writerSchema and resolves against readerSchema.

The client uses the decoder as if the data were written using readerSchema. // FIXME: Handle out of order fields.

◆ serialize()

template<typename Writer , typename T >

void avro::serialize	(	Writer &	s,
		const T &	val
	)

The main serializer entry point function.

Takes a serializer (either validating or plain) and the object that should be serialized.

◆ snapshot()

AVRO_DECL std::shared_ptr<std::vector<uint8_t> > avro::snapshot ( const OutputStream & source )

Returns the contents written so far into the output stream, which should be a memory output stream.

That is it must have been returned by a previous call to memoryOutputStream().

◆ toString()

const AVRO_DECL std::string& avro::toString ( Type type )

noexcept

Converts the given type into a string.

Useful for generating messages.

Variable Documentation

◆ ValidSchema

class AVRO_DECL avro::ValidSchema

This class is used to implement an avro spec parser using a flex/bison compiler.

In order for the lexer to be reentrant, this class provides a lexer object for each parse. The bison parser also uses this class to build up an avro parse tree as the avro spec is parsed.

Namespaces

Classes

Typedefs

Enumerations

Functions

Variables