2extrafield.md

A primitive column with extra field

Solution

This way you actually avoid serializing Enum value, instead you need to keep the actual column value to a primitive and dynamically instantiate Enum based on this value.

case class TestContainer(_v: String) {
  val v: EnumLike = EnumLike(_v)
}

Source

Pros

• 100% vanilla Scala Enum support

Cons

• extra field may be confusing
• primitive field can't be private (Spark limitation)
• overhead of creating extra objects

Source

• SPARK-17248
• Titanic Dataset