Master Theses Supervision

• Michaela Hörnfeldt, Safe Exploration in Reinforcement Learning: Safe Q-learning in a Grid-World, Master's Thesis in Computer and Systems Sciences, Stockholm University, Sweden, Main supervisor: Sindri Magnússon, Ph.D (June 2022).
• Omar Zia Toor, A Deep Reinforcement Learning Framework for Optimizing Fuel Economy of Vehicles, Master's Thesis in Artificial Intelligence, Stockholm University, Sweden, Main supervisor: Sindri Magnússon, Ph.D (June 2022).
• Simon Carlén, A Statistical and Machine Learning Approach to Air Pollution Forecasts, Master's Thesis in Artificial Intelligence, Stockholm University, Sweden, Main supervisor: Sindri Magnússon, Ph.D (September 2022).
• Mohammed Luqman, A comparative study of nature-inspired metaheuristic algorithms on sustainable road network planning, Master's Thesis in Artificial Intelligence, Stockholm University, Sweden, Main supervisor: Sindri Magnússon, Ph.D (September 2022).
• Bowen Meng, Deep reinforcement learning to the active control of ultra-low Reynolds number flows, Master's Thesis in Artificial Intelligence, Stockholm University, Sweden, Main supervisor: Sindri Magnússon, Ph.D (December 2022).
• Oscar Montilla Tabares, Predict failures and minimize costs based on sensor readings using deep learning, Master's Thesis in Computer and Systems Sciences, Stockholm University, Sweden, Main supervisor: Sindri Magnússon, Ph.D (June 2023).
• Daniella Blomberg, Claudio Bolzani, Reward machines for cooperative multi-agent reinforcement learning, Master's Thesis in Computer and Systems Sciences, Stockholm University, Sweden, Main supervisor: Sindri Magnússon, Ph.D (June 2023).
• Jingwen Zhao, Machine Learning-Based Treatment Recommendation for Patients with Obstructive Sleep Apnea, Joint Master's Thesis in Health Informatics, Karolinska Institutet & Stockholm University, Sweden, Main supervisor: Sindri Magnússon, Ph.D (July 2023).
• Laura Zubeidat, Towards gender fairness in machine learning algorithms, Joint Master's Thesis in Health Informatics, Karolinska Institutet & Stockholm University, Sweden, Main supervisor: Sindri Magnússon, Ph.D (July 2023).
• (Majid Hassanabadi), Bennet Voss, Collective AI Intelligence: From Single to Multi Agent Reinforcement Learning, Master's Thesis in Computer and Systems Sciences, Stockholm University, Sweden, Main supervisor: Sindri Magnússon, Ph.D (November 2023).
• (Biwen Zhu, Pedro Diniz), Fredrik Hammar, Benchmarking and Environment Design for Multi Agent Reinforcement Learning Algorithms, Master's Thesis in Computer and Systems Sciences, Stockholm University, Sweden, Main supervisor: Sindri Magnússon, Ph.D (February 2024).
• Ruchi Gupta, Danish Hashmi, Leveraging Machine Learning for optimal Order Lead Time prediction in Supply Chain Management, Master's Thesis in Computer and Systems Sciences, Stockholm University, Sweden, Main supervisor: Ali Beikmohammadi (May 2024).
• Yanjun Wang, Drug Discovery and Development by Reinforcement Learning, Joint Master's Thesis in Health Informatics, Karolinska Institutet & Stockholm University, Sweden, Main supervisor: Sindri Magnússon, Ph.D (May 2024).
• Michel Laji, Developing and Evaluating Naïve Transformer model for Seizure Detection on EEG, Joint Master's Thesis in Health Informatics, Karolinska Institutet & Stockholm University, Sweden, Main supervisor: Ali Beikmohammadi (May 2024).
• América Castrejón, Leaf Disease Detection Using Vision Transformers (ViT), Master's Thesis in Artificial Intelligence, Stockholm University, Sweden, Main supervisor: Ali Beikmohammadi (June 2024).
• Alfreds Lapkovskis, Natalia Nefedova, Advancements in Agriculture: Multimodal Deep Learning for Enhanced Plant Identification, Master's Thesis in Artificial Intelligence, Stockholm University, Sweden, Main supervisor: Ali Beikmohammadi (June 2024).
• Qingyu Huang, André Granberg, Federated Learning Empowers Agriculture: Collaborative Intelligence for Decentralized Crop Analysis and Management, Master's Thesis in Artificial Intelligence, Stockholm University, Sweden, Main supervisor: Ali Beikmohammadi (June 2024).
• Eric Hallberg, AI Integration: Exploring Technology and Human Challenges, Master's Thesis in Artificial Intelligence, Stockholm University, Sweden, Main supervisor: Ali Beikmohammadi (June 2024).
• Nikolaos Karampatzakis, Network routing using hierarchical reinforcement learning, Master's Thesis in Computer and Systems Sciences, Stockholm University, Sweden, Main supervisor: Sindri Magnússon, Ph.D (June 2024).
• Mahtab Babamohammadi, Accelerating Learning in Double Q-Learning: A Study on Speedy Double Q-learning for Efficient Reinforcement Learning, Master's Thesis in Artificial Intelligence, Stockholm University, Sweden, Main supervisor: Ali Beikmohammadi (September 2024).
• Mehdi Imani, Evaluating Classification and Sampling Methods for Customer Churn Prediction on Highly Imbalanced Data, Master's Thesis in Computer and Systems Sciences, Stockholm University, Sweden, Main supervisor: Ali Beikmohammadi (ongoing).
• Yulia Ryanova, Deep learning for classification of acute vestibular disorders, Master's Thesis in Computer and Systems Sciences, Stockholm University, Sweden, Main supervisor: Sindri Magnússon, Ph.D (ongoing).
• Wendy Mcrae, Multi Agent Reinforcement Learning: The Value of Cooperation and Communication, Master's Thesis in Computer and Systems Sciences, Stockholm University, Sweden, Main supervisor: Sindri Magnússon, Ph.D (ongoing).
• Juan Roijals Miras, fMRI correlates of drowsy brain states with implications to regime modeling, altered brain function and clinical prognosis, Joint Master's Thesis in Health Informatics, Karolinska Institutet & Stockholm University, Sweden, Main supervisor: Sindri Magnússon, Ph.D (ongoing).

Teaching Experience

• Current Research and Trends in Health Informatics
  • Fall 2023
  • Lecturer, M. Sc. Course, Joint Programme in Health Informatics, Karolinska Institutet & Stockholm University, Sweden.
• Current Research and Trends in Health Informatics
  • Fall 2022
  • Lecturer, M. Sc. Course, Joint Programme in Health Informatics, Karolinska Institutet & Stockholm University, Sweden.
• Machine Learning
  • Spring 2020
  • Teaching Assistant, M. Sc. Course, Department of Electrical Engineering, Amirkabir University of Technology, Tehran, Iran, under Sanaz Seyedin , Ph.D.
• Logical Circuits
  • Spring 2020
  • Teaching Assistant, M. Sc. Course, Department of Electrical Engineering, Amirkabir University of Technology, Tehran, Iran, under Professor Karim Faez.
• Digital Signal Processing
  • Fall 2019
  • Teaching Assistant, M. Sc. Course, Department of Electrical Engineering, Amirkabir University of Technology, Tehran, Iran, under Sanaz Seyedin, Ph.D.
• Microprocessors
  • Spring 2017
  • Teaching Assistant, B.Sc. Course, Department of Electrical Engineering, Bu-Ali Sina University, Hamedan, Iran, under Hamidreza Karami, Ph.D.
• Electronics I
  • Spring 2017
  • Teaching Assistant, B.Sc. Course, Department of Electrical Engineering, Bu-Ali Sina University, Hamedan, Iran, under Manouchehr Hosseini, Ph.D.
• Electronics II
  • Fall 2016
  • Teaching Assistant, B.Sc. Course, Department of Electrical Engineering, Bu-Ali Sina University, Hamedan, Iran, under Manouchehr Hosseini, Ph.D.
• Communication Systems I
  • Fall 2016
  • Teaching Assistant, B.Sc. Course, Department of Electrical Engineering, Bu-Ali Sina University, Hamedan, Iran, under Ali Kalantarnia, Ph.D.
• Electrical Circuits II
  • Spring 2016, Fall 2016
  • Teaching Assistant, B.Sc. Course, Department of Electrical Engineering, Bu-Ali Sina University, Hamedan, Iran, under Mohamadmahdi Shahbazi, Ph.D.
• Electromagnetism
  • Spring 2016
  • Teaching Assistant, B.Sc. Course, Department of Electrical Engineering, Bu-Ali Sina University, Hamedan, Iran, under Hamidreza Karami, Ph.D.
• Electrical Circuits I
  • Fall 2015
  • Teaching Assistant, B.Sc. Course, Department of Electrical Engineering, Bu-Ali Sina University, Hamedan, Iran, under Mohamadmahdi Shahbazi, Ph.D.

A selection from Stockholm University publication database

• Compressed Federated Reinforcement Learning with a Generative Model

  2024. Ali Beikmohammadi, Sarit Khirirat, Sindri Magnússon. Lecture Notes in Computer Science, 20-37

  Conference

  Reinforcement learning has recently gained unprecedented popularity, yet it still grapples with sample inefficiency. Addressing this challenge, federated reinforcement learning (FedRL) has emerged, wherein agents collaboratively learn a single policy by aggregating local estimations. However, this aggregation step incurs significant communication costs. In this paper, we propose CompFedRL, a communication-efficient FedRL approach incorporating both \textit{periodic aggregation} and (direct/error-feedback) compression mechanisms. Specifically, we consider compressed federated Q-learning with a generative model setup, where a central server learns an optimal Q-function by periodically aggregating compressed Q-estimates from local agents. For the first time, we characterize the impact of these two mechanisms (which have remained elusive) by providing a finite-time analysis of our algorithm, demonstrating strong convergence behaviors when utilizing either direct or error-feedback compression. Our bounds indicate improved solution accuracy concerning the number of agents and other federated hyperparameters while simultaneously reducing communication costs. To corroborate our theory, we also conduct in-depth numerical experiments to verify our findings, considering Top-K and Sparsified-K sparsification operators.

  Read more about Compressed Federated Reinforcement Learning with a Generative Model

• On the Convergence of Federated Learning Algorithms Without Data Similarity

  2024. Ali Beikmohammadi, Sarit Khirirat, Sindri Magnússon. IEEE Transactions on Big Data

  Article

  Data similarity assumptions have traditionally been relied upon to understand the convergence behaviors of federated learning methods. Unfortunately, this approach often demands fine-tuning step sizes based on the level of data similarity. When data similarity is low, these small step sizes result in an unacceptably slow convergence speed for federated methods. In this paper, we present a novel and unified framework for analyzing the convergence of federated learning algorithms without the need for data similarity conditions. Our analysis centers on an inequality that captures the influence of step sizes on algorithmic convergence performance. By applying our theorems to well-known federated algorithms, we derive precise expressions for three widely used step size schedules: fixed, diminishing, and step-decay step sizes, which are independent of data similarity conditions. Finally, we conduct comprehensive evaluations of the performance of these federated learning algorithms, employing the proposed step size strategies to train deep neural network models on benchmark datasets under varying data similarity conditions. Our findings demonstrate significant improvements in convergence speed and overall performance, marking a substantial advancement in federated learning research.

  Read more about On the Convergence of Federated Learning Algorithms Without Data Similarity

• Accelerating actor-critic-based algorithms via pseudo-labels derived from prior knowledge

  2024. Ali Beikmohammadi, Sindri Magnússon. Information Sciences 661

  Article

  Despite the huge success of reinforcement learning (RL) in solving many difficult problems, its Achilles heel has always been sample inefficiency. On the other hand, in RL, taking advantage of prior knowledge, intentionally or unintentionally, has usually been avoided, so that, training an agent from scratch is common. This not only causes sample inefficiency but also endangers safety –especially during exploration. In this paper, we help the agent learn from the environment by using the pre-existing (but not necessarily exact or complete) solution for a task. Our proposed method can be integrated with any RL algorithm developed based on policy gradient and actor-critic methods. The results on five tasks with different difficulty levels by using two well-known actor-critic-based methods as the backbone of our proposed method (SAC and TD3) show our success in greatly improving sample efficiency and final performance. We have gained these results alongside robustness to noisy environments at the cost of just a slight computational overhead, which is negligible.

  Read more about Accelerating actor-critic-based algorithms via pseudo-labels derived from prior knowledge

• TA-Explore: Teacher-Assisted Exploration for Facilitating Fast Reinforcement Learning

  2023. Ali Beikmohammadi, Sindri Magnússon. the 2023 International Conference on Autonomous Agents and Multiagent Systems, 2412-2414

  Conference

  Reinforcement Learning (RL) is crucial for data-driven decision-making but suffers from sample inefficiency. This poses a risk to system safety and can be costly in real-world environments with physical interactions. This paper proposes a human-inspired framework to improve the sample efficiency of RL algorithms, which gradually provides the learning agent with simpler but similar tasks that progress toward the main task. The proposed method does not require pre-training and can be applied to any goal, environment, and RL algorithm, including value-based and policy-based methods, as well as tabular and deep-RL methods. The framework is evaluated on a Random Walk and optimal control problem with constraint, showing good performance in improving the sample efficiency of RL-learning algorithms.

  Read more about TA-Explore: Teacher-Assisted Exploration for Facilitating Fast Reinforcement Learning

• Comparing NARS and Reinforcement Learning: An Analysis of ONA and Q-Learning Algorithms

  2023. Ali Beikmohammadi, Sindri Magnússon. Part of the Lecture Notes in Computer Science book series (LNAI,volume 13921), 21-31

  Conference

  In recent years, reinforcement learning (RL) has emerged as a popular approach for solving sequence-based tasks in machine learning. However, finding suitable alternatives to RL remains an exciting and innovative research area. One such alternative that has garnered attention is the Non-Axiomatic Reasoning System (NARS), which is a general-purpose cognitive reasoning framework. In this paper, we delve into the potential of NARS as a substitute for RL in solving sequence-based tasks. To investigate this, we conduct a comparative analysis of the performance of ONA as an implementation of NARS and Q-Learning in various environments that were created using the Open AI gym. The environments have different difficulty levels, ranging from simple to complex. Our results demonstrate that NARS is a promising alternative to RL, with competitive performance in diverse environments, particularly in non-deterministic ones.

  Read more about Comparing NARS and Reinforcement Learning: An Analysis of ONA and Q-Learning Algorithms

• SWP-LeafNET: A novel multistage approach for plant leaf identification based on deep CNN

  2022. Ali Beikmohammadi, Karim Faez, Ali Motallebi. Expert systems with applications 202

  Article

  Modern scientific and technological advances allow botanists to use computer vision-based approaches for plant identification tasks. These approaches have their own challenges. Leaf classification is a computer-vision task performed for the automated identification of plant species, a serious challenge due to variations in leaf morphology, including its size, texture, shape, and venation. Researchers have recently become more inclined toward deep learning-based methods rather than conventional feature-based methods due to the popularity and successful implementation of deep learning methods in image analysis, object recognition, and speech recognition.

  In this paper, to have an interpretable and reliable system, a botanist’s behavior is modeled in leaf identification by proposing a highly-efficient method of maximum behavioral resemblance developed through three deep learning-based models. Different layers of the three models are visualized to ensure that the botanist’s behavior is modeled accurately. The first and second models are designed from scratch. Regarding the third model, the pre-trained architecture MobileNetV2 is employed along with the transfer-learning technique. The proposed method is evaluated on two well-known datasets: Flavia and MalayaKew. According to a comparative analysis, the suggested approach is more accurate than hand-crafted feature extraction methods and other deep learning techniques in terms of 99.67% and 99.81% accuracy. Unlike conventional techniques that have their own specific complexities and depend on datasets, the proposed method requires no hand-crafted feature extraction. Also, it increases accuracy as compared with other deep learning techniques. Moreover, SWP-LeafNET is distributable and considerably faster than other methods because of using shallower models with fewer parameters asynchronously.

  Read more about SWP-LeafNET

• The Impact of SMOTE and ADASYN on Random Forest and Advanced Gradient Boosting Techniques in Telecom Customer Churn Prediction

  2024. Mehdi Imani (et al.). 2024 10th International Conference on Web Research (ICWR), 202-209

  Conference

  This paper explores the capability of various machine learning algorithms, including Random Forest and advanced gradient boosting techniques such as XGBoost, LightGBM, and CatBoost, to predict customer churn in the telecommunications sector. For this analysis, a dataset available to the public was employed. The performance of these algorithms was assessed using recognized metrics, including Accuracy, Precision, Recall, F1-score, and the Receiver Operating Characteristic Area Under Curve (ROC AUC). These metrics were evaluated at different phases: subsequent to data preprocessing and feature selection; following the application of SMOTE and ADASYN sampling methods; and after conducting hyperparameter tuning on the data that had been adjusted by SMOTE and ADASYN.The outcomes underscore the notable efficiency of upsampling techniques such as SMOTE and ADASYN in addressing the imbalance inherent in customer churn prediction. Notably, the application of random grid search for hyperparameter optimization did not significantly alter the results, which remained comparatively unchanged. The algorithms' performance post-ADASYN application marginally surpassed that observed after SMOTE application. Remarkably, LightGBM achieved an exceptional F1-score of 89% and an ROC AUC of 95% subsequent to the ADASYN sampling technique. This underlines the effectiveness of advanced boosting algorithms and upsampling methods like SMOTE and ADASYN in navigating the complexities of imbalanced datasets and intricate feature interdependencies.

  Read more about The Impact of SMOTE and ADASYN on Random Forest and Advanced Gradient Boosting Techniques in Telecom Customer Churn Prediction

Show all publications by Ali Beikmohammadi at Stockholm University