package discovery

Ordering
  1. Alphabetic
Visibility
  1. Public
  2. Protected

Package Members

  1. package schemas
  2. package spark
  3. package transformers
  4. package utils

Type Members

  1. final case class Anomaly(path: String, message: String, anomalyLevel: AnomalyLevel) extends Product with Serializable

    Anomalies are potential violations of a schema according to a given value.

    Anomalies are potential violations of a schema according to a given value.

    path

    the path where this anomaly occurred

    message

    a message describing this anomaly

    anomalyLevel

    the level of this anomaly

  2. sealed trait AnomalyLevel extends Ordered[AnomalyLevel]

    Represents different levels of anomaly severity.

  3. abstract class EquivalenceRelation extends Serializable

    Represents an equivalence relation between two schemas that determines whether two schemas should be considered equivalent in the final discovered schema.

    Represents an equivalence relation between two schemas that determines whether two schemas should be considered equivalent in the final discovered schema.

    For more details see [Parametric inference for massive JSON datasets](https://link.springer.com/article/10.1007/s00778-018-0532-7) by Baazizi et al

    Annotations
    @SerialVersionUID()
  4. final case class ForeignKey(localPath: String, foreignPath: String) extends Product with Serializable

    Represents a discovered foreign key.

    Represents a discovered foreign key.

    localPath

    the path of the referencing property

    foreignPath

    the path of the referenced property

  5. final case class Incompatibility[T](path: String, property: ClassTag[T]) extends Product with Serializable
  6. final case class JsonoidParams(additionalProperties: Boolean = false, er: EquivalenceRelation = EquivalenceRelations.KindEquivalenceRelation, extendedFormats: Boolean = false, formatThreshold: Float = 1.0f, maxExamples: Int = 100, propSet: PropertySet = PropertySets.AllProperties, resetFormatLength: Boolean = false) extends Product with Serializable

    Parameters used during the discovery process.

    Parameters used during the discovery process.

    additionalProperties

    whether additionalProperties is set to true in generated schemas

    er

    the equivalence relation to use during the discovery process

    extendedFormats

    whether to include extended formats

    formatThreshold

    the fraction of values that must match a given format for schemas.FormatProperty to consider the format valid

    maxExamples

    the maximum number of examples to be kept for any examples property

    resetFormatLength

    whether to reset max/min length of strings with schemas.FormatProperty

  7. sealed trait MergeType extends AnyRef

    The type of merge to be used when combining schemas.

  8. final case class PrimaryKey(path: String) extends Product with Serializable

    Represents a discovered primary key.

    Represents a discovered primary key.

    path

    the path of the primary key

  9. final case class PrimaryKeyFeatures(maxCount: Double, hasPrefixOrSuffix: Boolean, depth: Int, idType: Boolean, maxLength: Int) extends Product with Serializable

    A set of features used to score primary key candidates.

    A set of features used to score primary key candidates.

    maxCount

    the maximum cardinality of values at this path

    hasPrefixOrSuffix

    whether the path has a prefix or suffix common to primary keys

    depth

    the depth of the primary key in the JSON document

    idType

    the type of the primary key

    maxLength

    the maximum length of all prinary key values

  10. trait SchemaWalker[T] extends AnyRef

    A helper trait for any class that needs to walk a schema and build a map of values collected from the properties.

Value Members

  1. object AnomalyLevel
  2. case object BuildInfo extends Product with Serializable

    This object was generated by sbt-buildinfo.

  3. object DiscoverSchema
  4. object EquivalenceRelations

    Predefined sets of equivalence relations.

  5. object ForeignKeyFinder extends SchemaWalker[BloomFilter[_]]

    Find possible foreign keys by comparing values discovered from the schema.

  6. object Helpers

    Various helper functions used in the rest of the code.

  7. object IncompatibilityCollector
  8. case object Intersect extends MergeType with Product with Serializable

    Merge schemas by taking their intersection, producing a schema that will accept the all documents accepted by both schemas.

  9. object JsonoidParams extends Serializable
  10. object PrimaryKeyFeatures extends Serializable
  11. object PrimaryKeyFinder extends SchemaWalker[PrimaryKeyFeatures]
  12. object ReferenceResolver extends SchemaWalker[Unit]

    Resolve references in a schema by annotating all references with the schema that is being referenced.

    Resolve references in a schema by annotating all references with the schema that is being referenced. Specifically any instances of schemas.ReferencePointerProperty in schemas.ReferenceSchema will have an instance of schemas.ReferenceObjectProperty added.

  13. object TypeDetector

    Provides the ability to detect the schema type of a serialized JSON Schema object without type based on other available properties.

  14. case object Union extends MergeType with Product with Serializable

    Merge schemas by taking their union, producing a schema that will accept the all documents accepted by either schema.

  15. object ValueTableGenerator extends SchemaWalker[List[String]]

    Allow writing all values observed at a particular path to a CSV file.

Ungrouped