Live online course

ML Inference, Serving, and Monitoring

If you are already familiar with using general tools such as Docker and Kubernetes to organize and deploy your machine learning applications, then Module 3 of our MLOps learning track is specifically focused on serving machine learning models.

Duration: 6 live sessions start coming soon
Price: early birds €650 (full price €800)
* Get reimbursed by your employer

ML engineers, data scientists, data engineers, DevOps, and software engineers

By enrolling, you will:

learn how to use Seldon-Core for ML serving in various use cases
configure logging, monitoring, and metrics for your ML service
get experience with aNN libraries to deploy vector-based ML apps

6 live session

Immediate expert input

Real-world skills

You already know how to apply general tools like Docker and Kubernetes to structure and deploy your ML apps. Module 3 of our MLOps program is dedicated to serving ML models.

We’ll explore the capabilities of dedicated ML serving frameworks, learn how to deploy models as real-time services, build complex inference graphs, run A/B tests, and collect and process metrics.

Course syllabus

Session 1

Serving ML in production

TBA

ML inference and optimizations

ML serving vs regular APIs - what is the difference?

ML serving via Docker and Kubernetes

ML serving frameworks

Session 2

Seldon-Core

TBA

Architecture and use-cases

Standalone and k8s deployment

Seldon Custom Resource Definition (CRD)

Custom server and prebuilt servers: Sklearn, XGboost, MLFlow

Session 3

Advanced Seldon-Core

TBA

Seldon-Core Python components: model, transformer, aggregator, router

Seldon-core inference graphs

How to run AB-tests with Seldon-Core

Seldon-core feedback loops

Seldon-Deploy features

Session 4

Monitoring for ML

TBA

Prometheus for k8s services monitoring

Jaeger for tracing and APM

OpenTracing and OpenTelemetry

Grafana for visualizations

Online and offline metrics for ML services

How to monitor Seldon-Core services

Session 5

Efficient real-time aNN (Approximate Nearest Neighbours) search

TBA

aNN vs kNN

aNN algorithms and libraries

How to evaluate and optimize aNN: benchmarks and insights

How to deploy and manage aNN: platforms and custom solutions

Session 6

Demo day

TBA

Present team projects

Retrospective and follow-up questions

DMYTROVOITEKH

CTO

Lead ML Engineer

Meet the top expert

Dmytro Voitekh is a seasoned ML engineer with 8 years of experience in machine learning, MLOps, and full-stack engineering. He has a strong background in helping both early-stage startups and established tech companies to leverage ML features. His domains of expertise include machine learning and MLOps, and he has worked as an ML consultant, ML architect/lead, and CTO of a startup.

Dmytro has worked with a number of companies, including Proxet and GIPHY, and his extensive experience in the field makes him a valuable asset for any company that wants to incorporate machine learning into their business processes. He has numerous talks, workshops, and practical courses dedicated to ML, which showcases his expertise in the field.