Skip to content

[Feature Request]: Add Native Data Validation Mechanism to Apache Beam #37405

@asif6544

Description

@asif6544

What would you like to happen?

Add a native data validation mechanism to Apache Beam that allows developers to define validation rules at the schema or pipeline level, automatically handle invalid records (such as via side outputs or error sinks), and reduce the need for custom validation logic inside DoFns.

]

Image

Issue Priority

Priority: 3 (nice-to-have improvement)

Issue Components

  • Component: Python SDK
  • Component: Java SDK
  • Component: Go SDK
  • Component: Typescript SDK
  • Component: IO connector
  • Component: Beam YAML
  • Component: Beam examples
  • Component: Beam playground
  • Component: Beam katas
  • Component: Website
  • Component: Infrastructure
  • Component: Spark Runner
  • Component: Flink Runner
  • Component: Samza Runner
  • Component: Twister2 Runner
  • Component: Hazelcast Jet Runner
  • Component: Google Cloud Dataflow Runner

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions