Master Theses Supervision

• Michaela Hörnfeldt, Safe Exploration in Reinforcement Learning: Safe Q-learning in a Grid-World, Master's Thesis in Computer and Systems Sciences, Stockholm University, Sweden, Main supervisor: Sindri Magnússon, Ph.D (June 2022).
• Omar Zia Toor, A Deep Reinforcement Learning Framework for Optimizing Fuel Economy of Vehicles, Master's Thesis in Artificial Intelligence, Stockholm University, Sweden, Main supervisor: Sindri Magnússon, Ph.D (June 2022).
• Simon Carlén, A Statistical and Machine Learning Approach to Air Pollution Forecasts, Master's Thesis in Artificial Intelligence, Stockholm University, Sweden, Main supervisor: Sindri Magnússon, Ph.D (September 2022).
• Mohammed Luqman, A comparative study of nature-inspired metaheuristic algorithms on sustainable road network planning, Master's Thesis in Artificial Intelligence, Stockholm University, Sweden, Main supervisor: Sindri Magnússon, Ph.D (September 2022).
• Bowen Meng, Deep reinforcement learning to the active control of ultra-low Reynolds number flows, Master's Thesis in Artificial Intelligence, Stockholm University, Sweden, Main supervisor: Sindri Magnússon, Ph.D (December 2022).
• Oscar Montilla Tabares, Predict failures and minimize costs based on sensor readings using deep learning, Master's Thesis in Computer and Systems Sciences, Stockholm University, Sweden, Main supervisor: Sindri Magnússon, Ph.D (June 2023).
• Daniella Blomberg, Claudio Bolzani, Reward machines for cooperative multi-agent reinforcement learning, Master's Thesis in Computer and Systems Sciences, Stockholm University, Sweden, Main supervisor: Sindri Magnússon, Ph.D (June 2023).
• Jingwen Zhao, Machine Learning-Based Treatment Recommendation for Patients with Obstructive Sleep Apnea, Joint Master's Thesis in Health Informatics, Karolinska Institutet & Stockholm University, Sweden, Main supervisor: Sindri Magnússon, Ph.D (July 2023).
• Laura Zubeidat, Towards gender fairness in machine learning algorithms, Joint Master's Thesis in Health Informatics, Karolinska Institutet & Stockholm University, Sweden, Main supervisor: Sindri Magnússon, Ph.D (July 2023).
• (Majid Hassanabadi), Bennet Voss, Collective AI Intelligence: From Single to Multi Agent Reinforcement Learning, Master's Thesis in Computer and Systems Sciences, Stockholm University, Sweden, Main supervisor: Sindri Magnússon, Ph.D (November 2023).
• (Biwen Zhu, Pedro Diniz), Fredrik Hammar, Benchmarking and Environment Design for Multi Agent Reinforcement Learning Algorithms, Master's Thesis in Computer and Systems Sciences, Stockholm University, Sweden, Main supervisor: Sindri Magnússon, Ph.D (February 2024).
• Ruchi Gupta, Danish Hashmi, Leveraging Machine Learning for optimal Order Lead Time prediction in Supply Chain Management, Master's Thesis in Computer and Systems Sciences, Stockholm University, Sweden, Main supervisor: Ali Beikmohammadi (May 2024).
• Yanjun Wang, Drug Discovery and Development by Reinforcement Learning, Joint Master's Thesis in Health Informatics, Karolinska Institutet & Stockholm University, Sweden, Main supervisor: Sindri Magnússon, Ph.D (May 2024).
• Michel Laji, Developing and Evaluating Naïve Transformer model for Seizure Detection on EEG, Joint Master's Thesis in Health Informatics, Karolinska Institutet & Stockholm University, Sweden, Main supervisor: Ali Beikmohammadi (May 2024).
• América Castrejón, Leaf Disease Detection Using Vision Transformers (ViT), Master's Thesis in Artificial Intelligence, Stockholm University, Sweden, Main supervisor: Ali Beikmohammadi (June 2024).
• Alfreds Lapkovskis, Natalia Nefedova, Advancements in Agriculture: Multimodal Deep Learning for Enhanced Plant Identification, Master's Thesis in Artificial Intelligence, Stockholm University, Sweden, Main supervisor: Ali Beikmohammadi (June 2024).
• Qingyu Huang, André Granberg, Federated Learning Empowers Agriculture: Collaborative Intelligence for Decentralized Crop Analysis and Management, Master's Thesis in Artificial Intelligence, Stockholm University, Sweden, Main supervisor: Ali Beikmohammadi (June 2024).
• Eric Hallberg, AI Integration: Exploring Technology and Human Challenges, Master's Thesis in Artificial Intelligence, Stockholm University, Sweden, Main supervisor: Ali Beikmohammadi (June 2024).
• Nikolaos Karampatzakis, Network routing using hierarchical reinforcement learning, Master's Thesis in Computer and Systems Sciences, Stockholm University, Sweden, Main supervisor: Sindri Magnússon, Ph.D (June 2024).
• Mahtab Babamohammadi, Accelerating Learning in Double Q-Learning: A Study on Speedy Double Q-learning for Efficient Reinforcement Learning, Master's Thesis in Artificial Intelligence, Stockholm University, Sweden, Main supervisor: Ali Beikmohammadi (September 2024).
• Mehdi Imani, Evaluating Classification and Sampling Methods for Customer Churn Prediction on Highly Imbalanced Data, Master's Thesis in Computer and Systems Sciences, Stockholm University, Sweden, Main supervisor: Ali Beikmohammadi (ongoing).
• Yulia Ryanova, Deep learning for classification of acute vestibular disorders, Master's Thesis in Computer and Systems Sciences, Stockholm University, Sweden, Main supervisor: Sindri Magnússon, Ph.D (ongoing).
• Wendy Mcrae, Multi Agent Reinforcement Learning: The Value of Cooperation and Communication, Master's Thesis in Computer and Systems Sciences, Stockholm University, Sweden, Main supervisor: Sindri Magnússon, Ph.D (ongoing).
• Juan Roijals Miras, fMRI correlates of drowsy brain states with implications to regime modeling, altered brain function and clinical prognosis, Joint Master's Thesis in Health Informatics, Karolinska Institutet & Stockholm University, Sweden, Main supervisor: Sindri Magnússon, Ph.D (ongoing).

Teaching Experience

• Current Research and Trends in Health Informatics
  • Fall 2023
  • Lecturer, M. Sc. Course, Joint Programme in Health Informatics, Karolinska Institutet & Stockholm University, Sweden.
• Current Research and Trends in Health Informatics
  • Fall 2022
  • Lecturer, M. Sc. Course, Joint Programme in Health Informatics, Karolinska Institutet & Stockholm University, Sweden.
• Machine Learning
  • Spring 2020
  • Teaching Assistant, M. Sc. Course, Department of Electrical Engineering, Amirkabir University of Technology, Tehran, Iran, under Sanaz Seyedin , Ph.D.
• Logical Circuits
  • Spring 2020
  • Teaching Assistant, M. Sc. Course, Department of Electrical Engineering, Amirkabir University of Technology, Tehran, Iran, under Professor Karim Faez.
• Digital Signal Processing
  • Fall 2019
  • Teaching Assistant, M. Sc. Course, Department of Electrical Engineering, Amirkabir University of Technology, Tehran, Iran, under Sanaz Seyedin, Ph.D.
• Microprocessors
  • Spring 2017
  • Teaching Assistant, B.Sc. Course, Department of Electrical Engineering, Bu-Ali Sina University, Hamedan, Iran, under Hamidreza Karami, Ph.D.
• Electronics I
  • Spring 2017
  • Teaching Assistant, B.Sc. Course, Department of Electrical Engineering, Bu-Ali Sina University, Hamedan, Iran, under Manouchehr Hosseini, Ph.D.
• Electronics II
  • Fall 2016
  • Teaching Assistant, B.Sc. Course, Department of Electrical Engineering, Bu-Ali Sina University, Hamedan, Iran, under Manouchehr Hosseini, Ph.D.
• Communication Systems I
  • Fall 2016
  • Teaching Assistant, B.Sc. Course, Department of Electrical Engineering, Bu-Ali Sina University, Hamedan, Iran, under Ali Kalantarnia, Ph.D.
• Electrical Circuits II
  • Spring 2016, Fall 2016
  • Teaching Assistant, B.Sc. Course, Department of Electrical Engineering, Bu-Ali Sina University, Hamedan, Iran, under Mohamadmahdi Shahbazi, Ph.D.
• Electromagnetism
  • Spring 2016
  • Teaching Assistant, B.Sc. Course, Department of Electrical Engineering, Bu-Ali Sina University, Hamedan, Iran, under Hamidreza Karami, Ph.D.
• Electrical Circuits I
  • Fall 2015
  • Teaching Assistant, B.Sc. Course, Department of Electrical Engineering, Bu-Ali Sina University, Hamedan, Iran, under Mohamadmahdi Shahbazi, Ph.D.

Journal Publications and Preprints

• Ali Beikmohammadi, Sarit Khirirat, and Sindri Magnússon. "Distributed Momentum Methods Under Biased Gradient Estimations." Under Review in IEEE Transactions on Control of Network Systems. (link)
• Ali Beikmohammadi, and Sindri Magnússon. "Human-inspired framework to accelerate reinforcement learning." Under Review in Information Sciences. (link)
• Ali Beikmohammadi, Mohammad Hosein Hamian, Neda Khoeyniha, Tony Lindgren, Olof Steinert, and Sindri Magnússon. "A Cost-Sensitive Transformer Model for Prognostics Under Highly Imbalanced Industrial Data." Under Review in Engineering Applications of Artificial Intelligence. (link)
• Alfreds Lapkovskis, Natalia Nefedova, and Ali Beikmohammadi. "Automatic Fused Multimodal Deep Learning for Plant Identification." Under Review in Computers and Electronics in Agriculture. (link)
• Ali Beikmohammadi, Sarit Khirirat, and Sindri Magnússon. "On the Convergence of Federated Learning Algorithms without Data Similarity." Accepted in IEEE Transactions on Big Data, 2024. (link)
• Ali Beikmohammadi, and Sindri Magnússon. "Accelerating actor-critic-based algorithms via pseudo-labels derived from prior knowledge." Information Sciences, 2024. (link)
• Ali Beikmohammadi, Karim Faez, and Ali Motallebi. "SWP-LeafNET: A novel multistage approach for plant leaf identification based on deep CNN." Expert Systems with Applications, 2022. (link)

Books

• Beikmohammadi, Ali, et al. "Statistical Pattern Recognition." Amirkabir University of Technology Press, (in preparing). [in Persian]
• Beikmohammadi, A., et al. "Introduction to Deep Learning." Naghoos Press, 2020 [Translated in Persian from Charniak, E. Introduction to Deep Learning, The MIT Press, 2019].

Peer-Reviewed Conference Publications

• Ali Beikmohammadi, Sarit Khirirat, and Sindri Magnússon. "Compressed Federated Reinforcement Learning with a Generative Model." European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases (ECML-PKDD). 2024. (link) [Selected in the top 24% of submissions]
• Mehdi Imani, Zahra Ghaderpour, Majid Joudaki, and Ali Beikmohammadi. "The Impact of SMOTE and ADASYN on Random Forest and Advanced Gradient Boosting Techniques in Telecom Customer Churn Prediction." 10th International Conference on Web Research (ICWR). IEEE, 2024. (link)
• Ali Beikmohammadi, and Sindri Magnússon. "TA-Explore: Teacher-assisted exploration for facilitating fast reinforcement learning." Proceedings of the 2023 International Conference on Autonomous Agents and Multiagent Systems (AAMAS). 2023. (link)
• Ali Beikmohammadi, and Sindri Magnússon. "Comparing NARS and Reinforcement Learning: An Analysis of ONA and Q-Learning Algorithms." International Conference on Artificial General Intelligence (AGI-23). 2023. (link)
• Ali Beikmohammadi, and Najmeh Zahabi. "A Hierarchical Method for Kannada-MNIST Classification Based on Convolutional Neural Networks." 26th International Computer Conference, Computer Society of Iran (CSICC). IEEE, 2021. (link)
• Mohammad Hosein Hamian, Ali Beikmohammadi, Ali Ahmadi, and Babak Nasersharif. "Semantic Segmentation of Autonomous Driving Images by the Combination of Deep Learning and Classical Segmentation." 26th International Computer Conference, Computer Society of Iran (CSICC). IEEE, 2021. (link)
• Ali Beikmohammadi, Karim Faez, Mohammad Hosein Mahmoodian, and Mohammad Hosein Hamian. "Mixture of Deep-based Representation and Shallow Classifiers to Recognize Human Activities." 5th Iranian Conference on Signal Processing and Intelligent Systems (ICSPIS). IEEE, 2019. (link) [Outstanding Student Paper Award]
• Ali Beikmohammadi, and Karim Faez. "Leaf Classification for Plant Recognition with Deep Transfer Learning." 4th Iranian Conference on Signal Processing and Intelligent Systems (ICSPIS). IEEE, 2018. (link)
• Mohammad Hosein Mahmoodian, Hassan Taheri Gazvini, Ali Beikmohammadi. "A Novel Solution to Reduce Energy Consumption and Economic Revenue and, Increase Quality of Service and Resource Utilization in Cloud Data Centers." 7th National Congress of New Findings of Iranian Electrical Engineering (IEEEC7). 2020. (link) [in Persian]
• Ali Beikmohammadi, and Hamidreza Karami. "A Review of License Plate Segmentation and Recognition Methods." The First International Conference of Electronic and Computer Engineering. 2016. (link) [in Persian]
• Ali Beikmohammadi, and Hamidreza Karami. "A Review of License Plate Detection Methods‏." The First International Conference of Electronic and Computer Engineering. 2016. (link) [in Persian]
• Ali Beikmohammadi. "Gate Fringe Capacitance Modeling for FinFETs Considering RSD and Metal Contact in Its Structure, By Using 3-D Model." YREC First National Conference on Electrical Engineering. 2016. (indexed link) [in Persian]

Selected Projects (2017-2021)

• Facial expression classification using facial landmarks extraction [Team Work]
• 3D MRI brain tumor segmentation using deep learning. [Team Work]
• Using DL to improve the surface material classification utilizing EEG signals. [Team Work]
• Human action recognition using transfer learning with deep representations. [course project - Deep Learning]
• A bimodal learning approach to assist multi-sensory effects synchronization. [course project - Neural Networks]
• Learning deep CNN denoiser prior for image restoration. [course project - Statistical Pattern Recognition]
• Speech enhancement with DL using kernel decomposition techniques. [Team Work]
• Improvement of separable non-local means algorithm for image denoising. [course project - Machine Vision]
• Modeling and elucidation of housing price using Time-aware Latent Hierarchical Model. [course project - Machine Learning]
• Implementation of machine learning algorithms (decision tree, Bayes, Naïve Bayes, SVM, …). [course project - Machine Learning]
• Optimal allocation of resources using deep reinforcement learning in cognitive femtocell radio networks. [Team Work]
• Random access techniques for data transmission over packet-switched radio channels (Aloha & CSMA). [course project - Data Communication Networks]
• An enhanced and secured RSA key generation scheme (ESRKGS). [course project - Advanced Communication Networks]
• A fast-iterative recursive least squares algorithm for Wiener model identification of highly nonlinear systems. [Team Work]
• Peripheral component interconnects express. [course project - Micro Processors]
• Enter the Pentium CPU protected mode and perform processing operations with FPU and MMX units. [course project - Micro Processors]
• Design of smart sensors to control  the greenhouse environments by ESP8285 (IoT) [Team Work]

I urval från Stockholms universitets publikationsdatabas

• Compressed Federated Reinforcement Learning with a Generative Model

  2024. Ali Beikmohammadi, Sarit Khirirat, Sindri Magnússon. Lecture Notes in Computer Science, 20-37

  Konferens

  Reinforcement learning has recently gained unprecedented popularity, yet it still grapples with sample inefficiency. Addressing this challenge, federated reinforcement learning (FedRL) has emerged, wherein agents collaboratively learn a single policy by aggregating local estimations. However, this aggregation step incurs significant communication costs. In this paper, we propose CompFedRL, a communication-efficient FedRL approach incorporating both \textit{periodic aggregation} and (direct/error-feedback) compression mechanisms. Specifically, we consider compressed federated Q-learning with a generative model setup, where a central server learns an optimal Q-function by periodically aggregating compressed Q-estimates from local agents. For the first time, we characterize the impact of these two mechanisms (which have remained elusive) by providing a finite-time analysis of our algorithm, demonstrating strong convergence behaviors when utilizing either direct or error-feedback compression. Our bounds indicate improved solution accuracy concerning the number of agents and other federated hyperparameters while simultaneously reducing communication costs. To corroborate our theory, we also conduct in-depth numerical experiments to verify our findings, considering Top-K and Sparsified-K sparsification operators.

  Läs mer om Compressed Federated Reinforcement Learning with a Generative Model

• On the Convergence of Federated Learning Algorithms Without Data Similarity

  2024. Ali Beikmohammadi, Sarit Khirirat, Sindri Magnússon. IEEE Transactions on Big Data

  Artikel

  Data similarity assumptions have traditionally been relied upon to understand the convergence behaviors of federated learning methods. Unfortunately, this approach often demands fine-tuning step sizes based on the level of data similarity. When data similarity is low, these small step sizes result in an unacceptably slow convergence speed for federated methods. In this paper, we present a novel and unified framework for analyzing the convergence of federated learning algorithms without the need for data similarity conditions. Our analysis centers on an inequality that captures the influence of step sizes on algorithmic convergence performance. By applying our theorems to well-known federated algorithms, we derive precise expressions for three widely used step size schedules: fixed, diminishing, and step-decay step sizes, which are independent of data similarity conditions. Finally, we conduct comprehensive evaluations of the performance of these federated learning algorithms, employing the proposed step size strategies to train deep neural network models on benchmark datasets under varying data similarity conditions. Our findings demonstrate significant improvements in convergence speed and overall performance, marking a substantial advancement in federated learning research.

  Läs mer om On the Convergence of Federated Learning Algorithms Without Data Similarity

• Accelerating actor-critic-based algorithms via pseudo-labels derived from prior knowledge

  2024. Ali Beikmohammadi, Sindri Magnússon. Information Sciences 661

  Artikel

  Despite the huge success of reinforcement learning (RL) in solving many difficult problems, its Achilles heel has always been sample inefficiency. On the other hand, in RL, taking advantage of prior knowledge, intentionally or unintentionally, has usually been avoided, so that, training an agent from scratch is common. This not only causes sample inefficiency but also endangers safety –especially during exploration. In this paper, we help the agent learn from the environment by using the pre-existing (but not necessarily exact or complete) solution for a task. Our proposed method can be integrated with any RL algorithm developed based on policy gradient and actor-critic methods. The results on five tasks with different difficulty levels by using two well-known actor-critic-based methods as the backbone of our proposed method (SAC and TD3) show our success in greatly improving sample efficiency and final performance. We have gained these results alongside robustness to noisy environments at the cost of just a slight computational overhead, which is negligible.

  Läs mer om Accelerating actor-critic-based algorithms via pseudo-labels derived from prior knowledge

• TA-Explore: Teacher-Assisted Exploration for Facilitating Fast Reinforcement Learning

  2023. Ali Beikmohammadi, Sindri Magnússon. the 2023 International Conference on Autonomous Agents and Multiagent Systems, 2412-2414

  Konferens

  Reinforcement Learning (RL) is crucial for data-driven decision-making but suffers from sample inefficiency. This poses a risk to system safety and can be costly in real-world environments with physical interactions. This paper proposes a human-inspired framework to improve the sample efficiency of RL algorithms, which gradually provides the learning agent with simpler but similar tasks that progress toward the main task. The proposed method does not require pre-training and can be applied to any goal, environment, and RL algorithm, including value-based and policy-based methods, as well as tabular and deep-RL methods. The framework is evaluated on a Random Walk and optimal control problem with constraint, showing good performance in improving the sample efficiency of RL-learning algorithms.

  Läs mer om TA-Explore: Teacher-Assisted Exploration for Facilitating Fast Reinforcement Learning

• Comparing NARS and Reinforcement Learning: An Analysis of ONA and Q-Learning Algorithms

  2023. Ali Beikmohammadi, Sindri Magnússon. Part of the Lecture Notes in Computer Science book series (LNAI,volume 13921), 21-31

  Konferens

  In recent years, reinforcement learning (RL) has emerged as a popular approach for solving sequence-based tasks in machine learning. However, finding suitable alternatives to RL remains an exciting and innovative research area. One such alternative that has garnered attention is the Non-Axiomatic Reasoning System (NARS), which is a general-purpose cognitive reasoning framework. In this paper, we delve into the potential of NARS as a substitute for RL in solving sequence-based tasks. To investigate this, we conduct a comparative analysis of the performance of ONA as an implementation of NARS and Q-Learning in various environments that were created using the Open AI gym. The environments have different difficulty levels, ranging from simple to complex. Our results demonstrate that NARS is a promising alternative to RL, with competitive performance in diverse environments, particularly in non-deterministic ones.

  Läs mer om Comparing NARS and Reinforcement Learning: An Analysis of ONA and Q-Learning Algorithms

• SWP-LeafNET: A novel multistage approach for plant leaf identification based on deep CNN

  2022. Ali Beikmohammadi, Karim Faez, Ali Motallebi. Expert systems with applications 202

  Artikel

  Modern scientific and technological advances allow botanists to use computer vision-based approaches for plant identification tasks. These approaches have their own challenges. Leaf classification is a computer-vision task performed for the automated identification of plant species, a serious challenge due to variations in leaf morphology, including its size, texture, shape, and venation. Researchers have recently become more inclined toward deep learning-based methods rather than conventional feature-based methods due to the popularity and successful implementation of deep learning methods in image analysis, object recognition, and speech recognition.

  In this paper, to have an interpretable and reliable system, a botanist’s behavior is modeled in leaf identification by proposing a highly-efficient method of maximum behavioral resemblance developed through three deep learning-based models. Different layers of the three models are visualized to ensure that the botanist’s behavior is modeled accurately. The first and second models are designed from scratch. Regarding the third model, the pre-trained architecture MobileNetV2 is employed along with the transfer-learning technique. The proposed method is evaluated on two well-known datasets: Flavia and MalayaKew. According to a comparative analysis, the suggested approach is more accurate than hand-crafted feature extraction methods and other deep learning techniques in terms of 99.67% and 99.81% accuracy. Unlike conventional techniques that have their own specific complexities and depend on datasets, the proposed method requires no hand-crafted feature extraction. Also, it increases accuracy as compared with other deep learning techniques. Moreover, SWP-LeafNET is distributable and considerably faster than other methods because of using shallower models with fewer parameters asynchronously.

  Läs mer om SWP-LeafNET

• The Impact of SMOTE and ADASYN on Random Forest and Advanced Gradient Boosting Techniques in Telecom Customer Churn Prediction

  2024. Mehdi Imani (et al.). 2024 10th International Conference on Web Research (ICWR), 202-209

  Konferens

  This paper explores the capability of various machine learning algorithms, including Random Forest and advanced gradient boosting techniques such as XGBoost, LightGBM, and CatBoost, to predict customer churn in the telecommunications sector. For this analysis, a dataset available to the public was employed. The performance of these algorithms was assessed using recognized metrics, including Accuracy, Precision, Recall, F1-score, and the Receiver Operating Characteristic Area Under Curve (ROC AUC). These metrics were evaluated at different phases: subsequent to data preprocessing and feature selection; following the application of SMOTE and ADASYN sampling methods; and after conducting hyperparameter tuning on the data that had been adjusted by SMOTE and ADASYN.The outcomes underscore the notable efficiency of upsampling techniques such as SMOTE and ADASYN in addressing the imbalance inherent in customer churn prediction. Notably, the application of random grid search for hyperparameter optimization did not significantly alter the results, which remained comparatively unchanged. The algorithms' performance post-ADASYN application marginally surpassed that observed after SMOTE application. Remarkably, LightGBM achieved an exceptional F1-score of 89% and an ROC AUC of 95% subsequent to the ADASYN sampling technique. This underlines the effectiveness of advanced boosting algorithms and upsampling methods like SMOTE and ADASYN in navigating the complexities of imbalanced datasets and intricate feature interdependencies.

  Läs mer om The Impact of SMOTE and ADASYN on Random Forest and Advanced Gradient Boosting Techniques in Telecom Customer Churn Prediction

Visa alla publikationer av Ali Beikmohammadi vid Stockholms universitet