For AI agents: the complete documentation index is available at https://docs.dataplatform.ovh.net/llms.txt, the full documentation bundle is available at https://docs.dataplatform.ovh.net/llms-full.txt, and this page is available as Markdown at https://docs.dataplatform.ovh.net/landing-page-tutorials-data-engineering.md.
  • 🇬🇧 English

    • Data ingestion & transformation

    Bring data in, then make it worth querying.

    From raw source to modelled table

    Ingestion comes first. Streaming from Apache Kafka connects a broker as a source and keeps a table current from a topic. The IoT fleet-monitoring pipeline is the longer worked example, taking device messages through to dashboards in two parts, and is the closest thing here to a full reference architecture.

    Transformation follows. The Python SDK tutorials show how a Custom action reads a table, changes it in a dataframe and writes it back. SQL transformation does the same work in SQL on the Trino engine. Data models with external datasets builds tables over data the platform never copies. Segmentation is the performance chapter: splitting one job into parallel tasks by date, attribute, file set or account, across seven short guides covering each mode.

    Read them in order for a full pipeline, or pick the one that matches the source you have.

    Stream data from Apache Kafka→

    Keep a table current from a topic.

    Build an IoT fleet-monitoring pipeline→

    Device messages through to dashboards, in two parts.

    Transform data using the Python SDK→

    Read a table, change it in a dataframe, write it back.

    SQL transformation→

    The same work in SQL on the Trino engine.

    Data models with external datasets→

    Model over data the platform never copies.

    Apply segmentation to your jobs→

    Split one job into parallel tasks, seven modes covered.

    Go further

    If you need training or technical assistance to implement our solutions, contact your sales representative or click on this link to get a quote and ask our Professional Services experts for a custom analysis of your project.

    Ask questions, give your feedback and interact directly with the team building the Data Platform on the dedicated Discord channel.

    If you need support with your OVHcloud services, create a request in our Help Centre.

    Join our community of users.