For AI agents: the complete documentation index is available at https://docs.dataplatform.ovh.net/llms.txt, the full documentation bundle is available at https://docs.dataplatform.ovh.net/llms-full.txt, and this page is available as Markdown at https://docs.dataplatform.ovh.net/landing-page-tutorials-data-engineering.md.
  • 🇬🇧 English

    • Data ingestion & transformation

    Bring data in, then make it worth querying.

    From raw source to modelled table

    Ingestion comes first. Streaming from Apache Kafka connects a broker as a source and keeps a table current from a topic. The IoT fleet-monitoring pipeline is the longer worked example, taking device messages through to dashboards in two parts, and is the closest thing here to a full reference architecture.

    Transformation follows. The Python SDK tutorials show how a Custom action reads a table, changes it in a dataframe and writes it back. SQL transformation does the same work in SQL on the Trino engine. Data models with external datasets builds tables over data the platform never copies. Segmentation is the performance chapter: splitting one job into parallel tasks by date, attribute, file set or account, across seven short guides covering each mode.

    Read them in order for a full pipeline, or pick the one that matches the source you have.

    Stream data from Apache Kafka→

    Keep a table current from a topic.

    Build an IoT fleet-monitoring pipeline→

    Device messages through to dashboards, in two parts.

    Transform data using the Python SDK→

    Read a table, change it in a dataframe, write it back.

    SQL transformation→

    The same work in SQL on the Trino engine.

    Data models with external datasets→

    Model over data the platform never copies.

    Apply segmentation to your jobs→

    Split one job into parallel tasks, seven modes covered.

    Go further

    For training or technical assistance implementing our solutions, contact your sales representative or visit our Professional Services page to request a quote and have your project analysed by our experts.

    Ask questions, give your feedback and interact directly with the team building the Data Platform on the dedicated Discord channel.

    If you need support with your OVHcloud services, create a request in our Help Centre.

    Join our community of users.