Module serde_data_model_to_avro 

Mapping the Serde data model to the Avro data model

When manually mapping Rust types to an Avro schema, or the reverse, it is important to understand how the different data models are mapped. When mapping from an Avro schema to Rust types, see the documentation for the reverse.

Only the schemas generated by the AvroSchema derive and the mapping as defined here are supported. Any other behavior might change in a minor version.

The following list is based on the data model defined by Serde:

• 14 primitive types
  • bool => Schema::Boolean
  • i8, i16, i32, u8, u16 => Schema::Int
  • i64, u32 => Schema::Long
  • u64 => Schema::Fixed(name: "u64", size: 8)
    • This is not a Schema::Long as that is a signed number of maximum 64 bits.
  • i128 => Schema::Fixed(name: "i128", size: 16)
  • u128 => Schema::Fixed(name: "u128", size: 16)
  • f32 => Schema::Float
  • f64 => Schema::Double
  • char => Schema::String
    • Only one character allowed, deserializer will return an error for strings with more than one character.
• string => Schema::String
• byte array => Schema::Bytes or Schema::Fixed
• option => Schema::Union([Schema::Null, _])
• unit => Schema::Null
• unit struct => Schema::Record with the unqualified name equal to the name of the struct and zero fields
• unit variant => See Enums
• newtype struct => Schema::Record with the unqualified name equal to the name of the struct and one field
• newtype variant => See Enums
• seq => Schema::Array
• tuple
  • For tuples with one element, the schema will be the schema of the only element
  • For tuples with more than one element, the schema will be a Schema::Record with as many fields as there are elements. The schema must have the attribute org.apache.avro.rust.tuple with the value set to true.
  • Note: Serde (de)serializes arrays ([T; N]) as tuples. To (de)serialize an array as a Schema::Array use apache_avro::serde::array.
• tuple_struct => Schema::Record with the unqualified name equal to the name of the struct and as many fields as there are elements
  • Note: Tuple structs with 0 or 1 elements will also be (de)serialized as a Schema::Record. This is different from normal tuples.
• tuple_variant => See Enums
• map => Schema::Map
  • Note: the key type of the map will be (de)serialized as a Schema::String
• struct => Schema::Record
• struct_variant => See Enums

Enums

Externally tagged

This is the default enum representation for Serde. It can be mapped in three ways to the Avro data model. For all three options it is important that the enum index matches the Avro index.

• As a Schema::Enum, this is only possible for enums with only unit variants.
• As a Schema::Union with a Schema::Record for every variant:
  • unit_variant => Schema::Record named as the variant and with no fields.
  • newtype_variant => Schema::Record named as the variant and with one field. The schema must have the attribute org.apache.avro.rust.union_of_records with the value set to true.
  • tuple_variant => Schema::Record named as the variant and with as many fields as there are elements.
  • struct_variant => Schema::Record named as the variant and with a field for every field of the struct variant.
• As a Schema::Union without the wrapper Schema::Record, all schemas must be unique:
  • unit_variant => Schema::Null.
  • newtype_variant => The schema of the inner type.
  • tuple_variant => Schema::Record named as the variant and with as many fields as there are elements.
  • struct_variant => Schema::Record named as the variant and with a field for every field of the struct variant.

Internally tagged

This enum representation is used by Serde if the attribute #[serde(tag = "...")] is used. It maps to a Schema::Record. There must be at least one field that is named as the value of the tag attribute. If a field is not used by all variants it must have a default set.

Adjacently tagged

This enum representation is used by Serde if the attributes #[serde(tag = "...", content = "...")] are used. It maps to a Schema::Record with two fields. One field must be named as the value of the tag attribute and use the Schema::Enum schema. The other field must be named as the value of the content tag and use the Schema::Union schema.

Untagged

This enum representation is used by Serde if the attribute #[serde(untagged)] is used. It maps to a Schema::Union with the following schemas:

• unit_variant => Schema::Null.
• newtype_variant => The schema of the inner type.
• tuple_variant => Schema::Record named as the variant and with as many fields as there are elements.
• struct_variant => Schema::Record named as the variant and with a field for every field of the struct variant.