Zahra Alimoradzadeh, Master's thesis, M10

Date and time: Thursday 11/12, 8:30
Place: Cramér meeting room, Albano building 1
Student: Zahra Alimoradzadeh
Supervisor: Yishao Zhou
Title: "Asymptotic Analysis and Comparison of Model-Based and Model-Free Methods for the Linear Quadratic Regulator"

Abstract

This thesis studies the asymptotic sample efficiency of model-based and model-free reinforcement learning algorithms in the Linear Quadratic Regulator (LQR) setting. We focus on the problem of policy evaluation under a fixed linear controller, where the value function is quadratic and characterized by the unique solution P* of a discrete-time Lyapunov equation.
Two estimators of P* are analyzed:
   1- A model-based plug-in estimator, which estimates the closed-loop dynamics via regularized least squares and substitutes the estimate into the Lyapunov operator, and
   2- A model-free estimator based on Least-Squares Temporal Difference (LSTD) learning, which directly estimates the quadratic value function from trajectory data.
We analyze policy evaluation in infinite-horizon LQR under a fixed stabilizing controller, comparing a model-based plug-in estimator of the Lyapunov solution with a model-free LSTD estimator. Using Markov chain Central Limit Theorems (CLTs), the Delta Method, and uniform integrability, we establish that the model-based estimator attains strictly smaller asymptotic risk than LSTD.

Andreas Karlsson, Bachelor's thesis, K36

Date and time: Thursday 11/12, 13:00
Place: Mittag-Leffler meeting room, Albano building 1
Student: Andreas Karlsson
Supervisor: Annemarie Luger
Title: "Machine Learning in Kreĭn Spaces: An Exposition to the Main Representer Theorems"

Abstract

Kernel methods are powerful in machine learning because they enable nonlinear learning with simpler linear algebra tools. Yet most theory is built around positive semidefinite kernels and Hilbert-space geometry. This expository thesis explains how learning with indefinite kernels can be made rigorous and tractable in reproducing kernel Kreĭn spaces (RKKS). The thesis builds a concise pathway from the geometry of Kreĭn spaces (fundamental decomposition and symmetry, strong topology via the associated Hilbert space) to reproducing kernels, where evaluations are continuous and the kernel-induced form coincides with the RKKS inner product. On this foundation we synthesize two RKKS formulations of regularized empirical risk minimization: (1) a stabilization setup with a weak representer theorem, where stationary solutions lie in the data span, and (2) a variance-constrained minimization setup with a strong representer theorem, where optimizers lie in the span under strong-topology Tikhonov regularization. In both cases, the representer theorems make the infinite-dimensional problems collapse to finite-dimensional Gram-matrix algebra, introducing the kernel trick in the indefinite setting. A limitation of this setup is that naïve objectives can be unbounded below. This clarifies when indefinite kernels can be used reliably in specific supervised learning problems.

More presentations the same week

Maximilian Vranjes, Bachelor's thesis, K38

Date and time: Monday 8/12, 12:30
Place: Mittag-Leffler meeting room, Albano building 1
Student: Maximilian Vranjes
Supervisor: Anders Mörtberg
Title: "The Proofs of the Compactness Theorem of First Order Logic"

Simon Hanson, Bachelor's thesis, K35

Date and time: Monday 8/12, 14:00
Place: Cramér meeting room, Albano building 1
Student: Simon Hanson
Supervisor: Jonas Bergström
Title: "Hilbert's Invariance Theorem"

Anna Hall, Bachelor's thesis, K37

Date and time: Friday 12/12, 14:30
Place: Cramér meeting room, Albano building 1
Student: Anna Hall
Supervisor: Salvador Rodriguez Lopez
Title: "The Zeta function and The Prime Number Theorem"

For abstracts, see the department's calendar