All papers are listed below in reverse chronological order in which they appeared online.
Filters
All 2026 2025 2024 2023 2022 2021 2020 2019 2018 2017 2016 2015 2014 2013 2012 and earlier
All Acceleration Adaptive Asynchronous Broximal Compressed communication Coordinate descent Convex Decentralized Error feedback Federated learning First-order Linear algebra LLM Local training LoRA Minimax Momentum Muon Non-convex Primal-dual Privacy Proximal Second-order Sketching Stochastic Variance reduction Zero-order
Prepared in 2026
[318] Egor Shulgin, Tamaz Gadaev, Sarit Khirirat, and Peter Richtárik
Understanding MARS: when scaling momentum provably helps
43rd International Conference on Machine Learning (ICML 2026)
Algorithms:
Understanding MARS: when scaling momentum provably helps
43rd International Conference on Machine Learning (ICML 2026)
Algorithms:
MARS
[317] Egor Shulgin, Mohamed Awad, Peter Richtárik, and Eduard Gorbunov
General analysis of LMO-based optimizers: beyond bounded variance
43rd International Conference on Machine Learning (ICML 2026)
Algorithms:
General analysis of LMO-based optimizers: beyond bounded variance
43rd International Conference on Machine Learning (ICML 2026)
Algorithms:
NSGD with momentum, Muon
[316] Ivan Ilin, Philip Zmushko, and Peter Richtárik
Super-Tuning: from activation-aware pruning to sparse fine-tuning
arXiv github
Algorithms:
Super-Tuning: from activation-aware pruning to sparse fine-tuning
arXiv github
Algorithms:
Super, Supra
[315] Liyang Yuan, Yibo Yang, Dandan Guo, Peter Richtárik, and Zhouchen Lin
SpecGradFilter: a spectral gradient filtering framework for taming federated heterogeneity
arXiv
Algorithms:
SpecGradFilter: a spectral gradient filtering framework for taming federated heterogeneity
arXiv
Algorithms:
SpecGradFilter
[314] Xun Qian and Peter Richtárik
Convergence analysis of Muon-type methods with inexact LMO in the degenerate case
arXiv
Algorithms:
Convergence analysis of Muon-type methods with inexact LMO in the degenerate case
arXiv
Algorithms:
inexact Gluon, inexact Gluon with weight decay
[313] Igor Sokolov, Laurent Condat, and Peter Richtárik
SILAGE: memory-efficient, full-gradient-free nonconvex optimization for nested finite sums
arXiv
Algorithms:
SILAGE: memory-efficient, full-gradient-free nonconvex optimization for nested finite sums
arXiv
Algorithms:
SILAGE
[312] Ivan Ilin and Peter Richtárik
Demystifying pipeline parallelism: first theory for PipeDream
arXiv
Algorithms:
Demystifying pipeline parallelism: first theory for PipeDream
arXiv
Algorithms:
PipeDream, Randomized PipeDream
[311] Abdurakhmon Sadiev, Laurent Condat, and Peter Richtárik
A unified primal-dual recipe for accelerating three-operator splitting methods
arXiv
Algorithms:
A unified primal-dual recipe for accelerating three-operator splitting methods
arXiv
Algorithms:
ACV-I, ACV-II, APDTR-I, APDTR-II
[310] Yassine Maziane, Ammar Mahran, Artavazd Maranjyan, and Peter Richtárik
LOSCAR-SGD: local SGD with communication-computation overlap and delay-corrected sparse model averaging
arXiv
Algorithms:
LOSCAR-SGD: local SGD with communication-computation overlap and delay-corrected sparse model averaging
arXiv
Algorithms:
LOSCAR-SGD
[309] Yury Demidovich, Abhishek Chakraborty, Grigory Malinovsky, Angelia Nedić, and Peter Richtárik
Distance-aware Muon: adaptive step scaling for normalized optimization
arXiv
Algorithms:
Distance-aware Muon: adaptive step scaling for normalized optimization
arXiv
Algorithms:
DA-Muon, SC-Muon, DF-Muon
[308] Abdurakhmon Sadiev, Artavazd Maranjyan, Ivan Ilin, and Peter Richtárik
Ringmaster LMO: asynchronous linear minimization oracle momentum method
arXiv
Algorithms:
Ringmaster LMO: asynchronous linear minimization oracle momentum method
arXiv
Algorithms:
Ringmaster LMO
[307] Ammar Mahran, Artavazd Maranjyan, and Peter Richtárik
Rescaled asynchronous SGD: optimal distributed optimization under data and system heterogeneity
arXiv
Algorithms:
Rescaled asynchronous SGD: optimal distributed optimization under data and system heterogeneity
arXiv
Algorithms:
Rescaled ASGD
[306] Zhirayr Tovmasyan, Artavazd Maranjyan, and Peter Richtárik
Rennala MVR: improved time complexity for parallel stochastic optimization via momentum-based variance reduction
arXiv
Algorithms:
Rennala MVR: improved time complexity for parallel stochastic optimization via momentum-based variance reduction
arXiv
Algorithms:
Rennala MVR
[305] Peter Richtárik, Kaja Gruntkowska, and Hanmin Li
Local LMO: constrained gradient optimization via a local linear minimization oracle
arXiv slides
Algorithms:
Local LMO: constrained gradient optimization via a local linear minimization oracle
arXiv slides
Algorithms:
Local LMO
[304] Kaja Gruntkowska, Hanmin Li, Xun Qian, and Peter Richtárik
Broximal alignment for global non-convex optimization
arXiv
Algorithms:
Broximal alignment for global non-convex optimization
arXiv
Algorithms:
BPM
[303] Xun Qian, Alexander Gaponov, Grigory Malinovsky, and Peter Richtárik
Communication-efficient Gluon in federated learning
arXiv
Algorithms:
Communication-efficient Gluon in federated learning
arXiv
Algorithms:
Compressed Gluon with Error Feedback and MVR
[302] Laurent Condat, Abdurakhmon Sadiev, and Peter Richtárik
A Nesterov-accelerated primal-dual splitting algorithm for convex nonsmooth optimization
arXiv
Algorithms:
A Nesterov-accelerated primal-dual splitting algorithm for convex nonsmooth optimization
arXiv
Algorithms:
APAPC
[301] Hanmin Li, Kaja Gruntkowska, and Peter Richtárik
Stabilized proximal point method via trust region control
arXiv
Algorithms:
Stabilized proximal point method via trust region control
arXiv
Algorithms:
TRPPM
[300] Rustem Islamov, Grigory Malinovsky, Alexander Gaponov, Aurelien Lucchi, Peter Richtárik, and Eduard Gorbunov
Byzantine-robust and differentially private federated optimization under weaker assumptions
42nd Conference on Uncertainty in Artificial Intelligence (UAI 2026)
arXiv
Algorithms:
Byzantine-robust and differentially private federated optimization under weaker assumptions
42nd Conference on Uncertainty in Artificial Intelligence (UAI 2026)
arXiv
Algorithms:
Byz-Clip21-SGD2M
[299] Laurent Condat, Artavazd Maranjyan, and Peter Richtárik
BiCoLoR: Communication-efficient optimization with bidirectional compression and local training
arXiv
Algorithms:
BiCoLoR: Communication-efficient optimization with bidirectional compression and local training
arXiv
Algorithms:
BiCoLoR
Prepared in 2025
[298] Egor Shulgin, Grigory Malinovsky, Sarit Khirirat, and Peter Richtárik
First provable guarantees for practical private FL: beyond restrictive assumptions
arXiv
Algorithms:
First provable guarantees for practical private FL: beyond restrictive assumptions
arXiv
Algorithms:
Fed-α-NormEC, DP-Fed-α-NormEC
[297] Adrien Fradin, Abdurakhmon Sadiev, Laurent Condat, and Peter Richtárik
Tight lower bounds and optimal algorithms for stochastic nonconvex optimization with heavy-tailed noise
28th International Conference on Artificial Intelligence and Statistics (AISTATS 2026)
arXiv poster
Algorithms:
Tight lower bounds and optimal algorithms for stochastic nonconvex optimization with heavy-tailed noise
28th International Conference on Artificial Intelligence and Statistics (AISTATS 2026)
arXiv poster
Algorithms:
NSGD-MVR, NSGD-Hess, D-clip-NSGD-MVR, Clipped NSGD-Hess, SGD-MVR
[296] Xun Qian, Hussein Rammal, Dmitry Kovalev, and Peter Richtárik
Muon is provably faster with momentum variance reduction
arXiv
Algorithms:
Muon is provably faster with momentum variance reduction
arXiv
Algorithms:
Muon-MVR, Gluon-MVR-1, Gluon-MVR-2, Gluon-MVR-3
[295] Sarit Khirirat, Abdurakhmon Sadiev, Yury Demidovich, and Peter Richtárik
Better LMO-based momentum methods with second-order information
arXiv
Algorithms:
Better LMO-based momentum methods with second-order information
arXiv
Algorithms:
LMO-SOM
[294] Abdurakhmon Sadiev, Yury Demidovich, Igor Sokolov, Grigory Malinovsky, Sarit Khirirat, and Peter Richtárik
Improved convergence in parameter-agnostic error feedback through momentum
arXiv
Algorithms:
Improved convergence in parameter-agnostic error feedback through momentum
arXiv
Algorithms:
‖ EF21-SGDM ‖, ‖ EF21-IGT ‖, ‖ EF21-RHM ‖, ‖ EF21-HM ‖, ‖ EF21-MVR ‖
[293] Egor Shulgin, Sultan AlRashed, Francesco Orabona, and Peter Richtárik
Beyond the ideal: Analyzing the inexact Muon update
28th International Conference on Artificial Intelligence and Statistics (AISTATS 2026)
arXiv
Algorithms:
Beyond the ideal: Analyzing the inexact Muon update
28th International Conference on Artificial Intelligence and Statistics (AISTATS 2026)
arXiv
Algorithms:
Muon
[292] Abdurakhmon Sadiev, Peter Richtárik, and Ilyas Fatkhullin
Second-order optimization under heavy-tailed noise: Hessian clipping and sample complexity limits
Advances in Neural Information Processing Systems 39 (NeurIPS 2025)
arXiv poster
Algorithms:
Second-order optimization under heavy-tailed noise: Hessian clipping and sample complexity limits
Advances in Neural Information Processing Systems 39 (NeurIPS 2025)
arXiv poster
Algorithms:
NSGDHess, Clip-NSGDHess
[291] Kaja Gruntkowska, Yassine Maziane, Zheng Qu, and Peter Richtárik
Drop-Muon: Update less, converge faster
arXiv
Algorithms:
Drop-Muon: Update less, converge faster
arXiv
Algorithms:
Drop-Muon
[290] Kaja Gruntkowska and Peter Richtárik
Non-Euclidean broximal point method: a blueprint for geometry-aware optimization
arXiv
Algorithms:
Non-Euclidean broximal point method: a blueprint for geometry-aware optimization
arXiv
Algorithms:
BPM
[289] Kaja Gruntkowska, Alexander Gaponov, Zhirayr Tovmasyan, and Peter Richtárik
Error feedback for Muon and friends
14th International Conference on Learning Representations (ICLR 2026)
arXiv poster
Algorithms:
Error feedback for Muon and friends
14th International Conference on Learning Representations (ICLR 2026)
arXiv poster
Algorithms:
EF21-Muon
[288] Adrien Fradin, Peter Richtárik, and Alexander Tyurin
Local SGD and federated averaging through the lens of time complexity
arXiv
Algorithms:
Local SGD and federated averaging through the lens of time complexity
arXiv
Algorithms:
Dual Local SGD, Decaying Local SGD, Decaying Local ASGD
[287] Artavazd Maranjyan and Peter Richtárik
Ringleader ASGD: The first asynchronous SGD with optimal time complexity under data heterogeneity
14th International Conference on Learning Representations (ICLR 2026)
arXiv poster slides
Algorithms:
Ringleader ASGD: The first asynchronous SGD with optimal time complexity under data heterogeneity
14th International Conference on Learning Representations (ICLR 2026)
arXiv poster slides
Algorithms:
Ringleader ASGD
[286] Laurent Condat and Peter Richtárik
Convergence Analysis of the ProbAbilistic Gradient Estimator Algorithm for Weakly Convex Finite-Sum Optimization
To Appear In: Journal of Optimization Theory and Applications
arXiv
Algorithms:
Convergence Analysis of the ProbAbilistic Gradient Estimator Algorithm for Weakly Convex Finite-Sum Optimization
To Appear In: Journal of Optimization Theory and Applications
arXiv
Algorithms:
PAGE
[285] Igor Sokolov, Abdurakhmon Sadiev, Yury Demidovich, Fawaz S Al-Qahtani, and Peter Richtárik
Bernoulli-LoRA: A theoretical framework for randomized low-rank adaptation
arXiv
Algorithms:
Bernoulli-LoRA: A theoretical framework for randomized low-rank adaptation
arXiv
Algorithms:
Bernoulli-LoRA
[284] Artem Riabinin, Egor Shulgin, Kaja Gruntkowska, and Peter Richtárik
From Muon to Gluon: bridging theory and practice of LMO-based optimizers for LLMs
43rd International Conference on Machine Learning (ICML 2026)
arXiv slides
Algorithms:
From Muon to Gluon: bridging theory and practice of LMO-based optimizers for LLMs
43rd International Conference on Machine Learning (ICML 2026)
arXiv slides
Algorithms:
Gluon, Muon, Scion
[283] Laurent Condat, Elnur Gasanov, and Peter Richtárik
The stochastic multi-proximal method for nonsmooth optimization
arXiv
Algorithms:
The stochastic multi-proximal method for nonsmooth optimization
arXiv
Algorithms:
SMPM, FedSMPM, Point-SAGA, ProxSkip, Davis-Yin
[282] Ivan Ilin and Peter Richtárik
Thanos: a block-wise pruning algorithm for efficient large language model compression
arXiv
Algorithms:
Thanos: a block-wise pruning algorithm for efficient large language model compression
arXiv
Algorithms:
Thanos
[281] Ali Beikmohammadi, Sarit Khirirat, Peter Richtárik, and Sindri Magnússon
Collaborative value function estimation under model mismatch: a federated temporal difference analysis
Machine Learning and Knowledge Discovery in Databases. Research Track (ECML PKDD 2025)
arXiv
Algorithms:
Collaborative value function estimation under model mismatch: a federated temporal difference analysis
Machine Learning and Knowledge Discovery in Databases. Research Track (ECML PKDD 2025)
arXiv
Algorithms:
FedTD (0)
[280] Konstantin Burlachenko and Peter Richtárik
BurTorch: Revisiting training from first principles by coupling autodiff, math optimization, and systems
arXiv
Algorithms:
BurTorch: Revisiting training from first principles by coupling autodiff, math optimization, and systems
arXiv
Algorithms:
BurTorch
[279] Egor Shulgin, Sarit Khirirat, and Peter Richtárik
Smoothed normalization for efficient distributed private optimization
arXiv poster
Algorithms:
Smoothed normalization for efficient distributed private optimization
arXiv poster
Algorithms:
α-𝖭𝗈𝗋𝗆𝖤𝖢
[278] Artem Riabinin, Ahmed Khaled, and Peter Richtárik
A novel unified parametric assumption for nonconvex optimization
arXiv
Algorithms:
A novel unified parametric assumption for nonconvex optimization
arXiv
Algorithms:
GD, SGD
[277] Rustem Islamov, Samuel Horváth, Aurelien Lucchi, Peter Richtárik, and Eduard Gorbunov
Double momentum and error feedback for clipping with fast rates and differential privacy
arXiv
Algorithms:
Double momentum and error feedback for clipping with fast rates and differential privacy
arXiv
Algorithms:
Clip21-SGD2M
[276] Zhirayr Tovmasyan, Grigory Malinovsky, Laurent Condat, and Peter Richtárik
Revisiting stochastic proximal point methods: generalized smoothness and similarity
Journal of Nonlinear and Variational Analysis 10(3):471-505, 2026
arXiv
Algorithms:
Revisiting stochastic proximal point methods: generalized smoothness and similarity
Journal of Nonlinear and Variational Analysis 10(3):471-505, 2026
arXiv
Algorithms:
SPPM
[275] Kaja Gruntkowska, Hanmin Li, Aadi Rane, and Peter Richtárik
The ball-proximal (="broximal") point method: a new algorithm, convergence theory, and applications
arXiv video slides
Algorithms:
The ball-proximal (="broximal") point method: a new algorithm, convergence theory, and applications
arXiv video slides
Algorithms:
BPM, ‖ PPM ‖
[274] Artavazd Maranjyan, El Mehdi Saad, Peter Richtárik, and Francesco Orabona
ATA: Adaptive task allocation for efficient resource management in distributed machine learning
42nd International Conference on Machine Learning (ICML 2025)
arXiv poster
Algorithms:
ATA: Adaptive task allocation for efficient resource management in distributed machine learning
42nd International Conference on Machine Learning (ICML 2025)
arXiv poster
Algorithms:
ATA
[273] Kai Yi and Peter Richtárik
Symmetric pruning of large language models
arXiv poster
Algorithms:
Symmetric pruning of large language models
arXiv poster
Algorithms:
Symmetric Wanda, R2-DSnoT
[272] Artavazd Maranjyan, Alexander Tyurin and Peter Richtárik
Ringmaster ASGD: The first asynchronous SGD with optimal time complexity
42nd International Conference on Machine Learning (ICML 2025)
arXiv poster
Algorithms:
Ringmaster ASGD: The first asynchronous SGD with optimal time complexity
42nd International Conference on Machine Learning (ICML 2025)
arXiv poster
Algorithms:
Naive Optimal ASGD, Ringmaster ASGD
Prepared in 2024
[271] Egor Shulgin and Peter Richtárik
On the convergence of DP-SGD with adaptive clipping
arXiv
Algorithms:
On the convergence of DP-SGD with adaptive clipping
arXiv
Algorithms:
QC-SGD, DP-QC-SGD
[270] Igor Sokolov and Peter Richtárik
MARINA-P: Superior performance in non-smooth federated optimization with adaptive stepsizes
arXiv
Algorithms:
MARINA-P: Superior performance in non-smooth federated optimization with adaptive stepsizes
arXiv
Algorithms:
MARINA-P
[269] Artavazd Maranjyan, Abdurakhmon Sadiev, and Peter Richtárik
Differentially private random block coordinate descent
arXiv
Algorithms:
Differentially private random block coordinate descent
arXiv
Algorithms:
DP-SkGD, DP-SkGD-BS, DP-CD
[268] Elnur Gasanov and Peter Richtárik
Speeding up stochastic proximal optimization in the high Hessian dissimilarity setting
arXiv
Algorithms:
Speeding up stochastic proximal optimization in the high Hessian dissimilarity setting
arXiv
Algorithms:
L-SVRP
[267] Yury Demidovich, Petr Ostroukhov, Grigory Malinovsky, Samuel Horváth, Martin Takáč, Peter Richtárik, and Eduard Gorbunov
Methods with local steps and random reshuffling for generally smooth non-convex federated optimization
13th International Conference on Learning Representations (ICLR 2025)
arXiv poster
Algorithms:
Methods with local steps and random reshuffling for generally smooth non-convex federated optimization
13th International Conference on Learning Representations (ICLR 2025)
arXiv poster
Algorithms:
Clip-LocalGDJ, CLERR, Clipped RR-CLI
[266] Vladimir Malinovskii, Andrei Panferov, Ivan Ilin, Han Guo, Peter Richtárik, and Dan Alistarh
Pushing the limits of large language model quantization via the linearity theorem
The 2025 Annual Conference of the Nations of the Americas Chapter of the ACL (NAACL 2025)
arXiv
Algorithms:
Pushing the limits of large language model quantization via the linearity theorem
The 2025 Annual Conference of the Nations of the Americas Chapter of the ACL (NAACL 2025)
arXiv
Algorithms:
HIGGS
[265] Sarit Khirirat, Abdurakhmon Sadiev, Artem Riabinin, Eduard Gorbunov, and Peter Richtárik
Error feedback under $(L_0,L_1)$-smoothness: normalization and momentum
Advances in Neural Information Processing Systems 39 (NeurIPS 2025)
arXiv poster
Algorithms:
Error feedback under $(L_0,L_1)$-smoothness: normalization and momentum
Advances in Neural Information Processing Systems 39 (NeurIPS 2025)
arXiv poster
Algorithms:
‖ EF21‖, ‖ EF21-SGDM ‖
[264] Wojciech Anyszka, Kaja Gruntkowska, Alexander Tyurin, and Peter Richtárik
Tighter performance theory of FedExProx
14th International Conference on Learning Representations (ICLR 2026)
arXiv poster
Algorithms:
Tighter performance theory of FedExProx
14th International Conference on Learning Representations (ICLR 2026)
arXiv poster
Algorithms:
FedExProx
[263] Konstantin Burlachenko and Peter Richtárik
Unlocking FedNL: Self-contained compute-optimized implementation
arXiv
Algorithms:
Unlocking FedNL: Self-contained compute-optimized implementation
arXiv
Algorithms:
FedNL, FedNL-LS, FedNL-PP
[262] Grigory Malinovsky, Umberto Michieli, Hasan Abed Al Kader Hammoud, Taha Ceritli, Hayder Elesedy, Mete Ozay, and Peter Richtárik
Randomized asymmetric chain of LoRA: The first meaningful theoretical framework for low-rank adaptation
arXiv
Algorithms:
Randomized asymmetric chain of LoRA: The first meaningful theoretical framework for low-rank adaptation
arXiv
Algorithms:
RAC-LoRA, Fed-RAC-LoRA
[261] Artavazd Maranjyan, Omar Shaikh Omar, and Peter Richtárik
MindFlayer SGD: Efficient parallel SGD in the presence of heterogeneous and random worker compute times
41st Conference on Uncertainty in Artificial Intelligence (UAI 2025)
NeurIPS 2024 Workshop: Optimization for Machine Learning (OPT 2024)
to be presented at Conference on the Mathematical Theory of Deep Neural Networks (DeepMath 2024) arXiv poster
Algorithms:
MindFlayer SGD: Efficient parallel SGD in the presence of heterogeneous and random worker compute times
41st Conference on Uncertainty in Artificial Intelligence (UAI 2025)
NeurIPS 2024 Workshop: Optimization for Machine Learning (OPT 2024)
to be presented at Conference on the Mathematical Theory of Deep Neural Networks (DeepMath 2024) arXiv poster
Algorithms:
MindFlayer SGD, Vecna SGD
[260] Hanmin Li and Peter Richtárik
On the convergence of FedProx with extrapolation and inexact prox
NeurIPS 2024 Workshop: Optimization for Machine Learning (OPT 2024)
arXiv
Algorithms:
On the convergence of FedProx with extrapolation and inexact prox
NeurIPS 2024 Workshop: Optimization for Machine Learning (OPT 2024)
arXiv
Algorithms:
FedExProx
[259] Eduard Gorbunov, Nazarii Tupitsa, Sayantan Choudhury, Alen Aliev, Peter Richtárik, Samuel Horváth, and Martin Takáč
Methods for convex $(L_0,L_1)$-smooth optimization: clipping, acceleration, and adaptivity
13th International Conference on Learning Representations (ICLR 2025)
arXiv poster
Algorithms:
Methods for convex $(L_0,L_1)$-smooth optimization: clipping, acceleration, and adaptivity
13th International Conference on Learning Representations (ICLR 2025)
arXiv poster
Algorithms:
L0L1-GD, L0L1-GD-PS, L0L1-STM, L0L1-AdGD, L0L1-SGD, L0L1-SGD-PS
[258] Kai Yi, Timur Kharisov, Igor Sokolov, and Peter Richtárik
Cohort squeeze: Beyond a single communication round per cohort in cross-device federated learning
Oral at the NeurIPS 2024 Federated Learning Workshop
arXiv
Algorithms:
Cohort squeeze: Beyond a single communication round per cohort in cross-device federated learning
Oral at the NeurIPS 2024 Federated Learning Workshop
arXiv
Algorithms:
SPPM-AS
[257] Georg Meinhardt, Kai Yi, Laurent Condat, and Peter Richtárik
Sparse-ProxSkip: Accelerated sparse-to-sparse training in federated learning
arXiv
Algorithms:
Sparse-ProxSkip: Accelerated sparse-to-sparse training in federated learning
arXiv
Algorithms:
Sparse-ProxSkip
[256] Avetik Karagulyan, Egor Shulgin, Abdurakhmon Sadiev, and Peter Richtárik
SPAM: Stochastic proximal point method with momentum variance reduction for non-convex cross-device federated learning
NeurIPS 2024 Workshop: Optimization for Machine Learning (OPT 2024)
arXiv
Algorithms:
SPAM: Stochastic proximal point method with momentum variance reduction for non-convex cross-device federated learning
NeurIPS 2024 Workshop: Optimization for Machine Learning (OPT 2024)
arXiv
Algorithms:
SPAM
[255] Laurent Condat and Peter Richtárik
A simple linear convergence analysis of the Point-SAGA algorithm
arXiv
Algorithms:
A simple linear convergence analysis of the Point-SAGA algorithm
arXiv
Algorithms:
Point-SAGA
[254] Peter Richtárik, Simone Maria Giancola, Dymitr Lubczyk, and Robin Yadav
Local curvature descent: Squeezing more curvature out of standard and Polyak gradient descent
Advances in Neural Information Processing Systems 39 (NeurIPS 2025)
NeurIPS 2024 Workshop: Optimization for Machine Learning (OPT 2024)
arXiv poster
Algorithms:
Local curvature descent: Squeezing more curvature out of standard and Polyak gradient descent
Advances in Neural Information Processing Systems 39 (NeurIPS 2025)
NeurIPS 2024 Workshop: Optimization for Machine Learning (OPT 2024)
arXiv poster
Algorithms:
LCD1, LCD2, LCD3
[253] Alexander Tyurin and Peter Richtárik
On the optimal time complexities in decentralized stochastic asynchronous optimization
Advances in Neural Information Processing Systems 38 (NeurIPS 2024)
arXiv poster
Algorithms:
On the optimal time complexities in decentralized stochastic asynchronous optimization
Advances in Neural Information Processing Systems 38 (NeurIPS 2024)
arXiv poster
Algorithms:
Fragile SGD, Amelie SGD
[252] Peter Richtárik, Abdurakhmon Sadiev, and Yury Demidovich
A unified theory of stochastic proximal point methods without smoothness
arXiv
Algorithms:
A unified theory of stochastic proximal point methods without smoothness
arXiv
Algorithms:
SPPM, SPPM-LC, SPPM-NS, SPPM-AS, SPPM*, SPPM-GC, L-SVRP, Point SAGA
[251] Ionut-Vlad Modoranu, Mher Safaryan, Grigory Malinovsky, Eldar Kurtic, Thomas Robert, Peter Richtárik, and Dan Alistarh
MicroAdam: Accurate adaptive optimization with low space overhead and provable convergence
Advances in Neural Information Processing Systems 38 (NeurIPS 2024)
arXiv poster
Algorithms:
MicroAdam: Accurate adaptive optimization with low space overhead and provable convergence
Advances in Neural Information Processing Systems 38 (NeurIPS 2024)
arXiv poster
Algorithms:
MicroAdam
[250] Alexander Tyurin, Kaja Gruntkowska, and Peter Richtárik
Freya PAGE: First optimal time complexity for large-scale nonconvex finite-sum optimization with heterogeneous asynchronous computations
Advances in Neural Information Processing Systems 38 (NeurIPS 2024)
arXiv poster slides
Algorithms:
Freya PAGE: First optimal time complexity for large-scale nonconvex finite-sum optimization with heterogeneous asynchronous computations
Advances in Neural Information Processing Systems 38 (NeurIPS 2024)
arXiv poster slides
Algorithms:
Freya PAGE, Freya SGD
[249] Vladimir Malinovskii, Denis Mazur, Ivan Ilin, Denis Kuznedelev, Konstantin Burlachenko, Kai Yi, Dan Alistarh, and Peter Richtárik
PV-Tuning: Beyond straight-through estimation for extreme LLM compression
Advances in Neural Information Processing Systems 38 (NeurIPS 2024)
Oral at NeurIPS 2024 (0.4\% acceptance rate)
arXiv poster
Algorithms:
PV-Tuning: Beyond straight-through estimation for extreme LLM compression
Advances in Neural Information Processing Systems 38 (NeurIPS 2024)
Oral at NeurIPS 2024 (0.4\% acceptance rate)
arXiv poster
Algorithms:
PV
[248] Abdurakhmon Sadiev, Laurent Condat, and Peter Richtárik
Stochastic proximal point methods for monotone inclusions under expected similarity
NeurIPS 2024 Workshop: Optimization for Machine Learning (OPT 2024)
arXiv
Algorithms:
Stochastic proximal point methods for monotone inclusions under expected similarity
NeurIPS 2024 Workshop: Optimization for Machine Learning (OPT 2024)
arXiv
Algorithms:
SPPM, SPPM-OC, L-SVRP
[247] Hanmin Li, Kirill Acharya, and Peter Richtárik
The power of extrapolation in federated learning
Advances in Neural Information Processing Systems 38 (NeurIPS 2024)
arXiv poster
Algorithms:
The power of extrapolation in federated learning
Advances in Neural Information Processing Systems 38 (NeurIPS 2024)
arXiv poster
Algorithms:
FedExProx, FedExProx-GraDS, FedExProx-StoPS
[246] Kai Yi, Georg Meinhardt, Laurent Condat, and Peter Richtárik
FedComLoc: Communication-efficient distributed training of sparse and quantized models
Transactions on Machine Learning Research (TMLR 2025)
arXiv
Algorithms:
FedComLoc: Communication-efficient distributed training of sparse and quantized models
Transactions on Machine Learning Research (TMLR 2025)
arXiv
Algorithms:
FedComLoc
[245] Yury Demidovich, Grigory Malinovsky, and Peter Richtárik
Streamlining in the Riemannian realm: Efficient Riemannian optimization with loopless variance reduction
arXiv
Best Paper Award (runner-up), International Conference on Computational Optimization (ICOMP 2025)
Algorithms:
Streamlining in the Riemannian realm: Efficient Riemannian optimization with loopless variance reduction
arXiv
Best Paper Award (runner-up), International Conference on Computational Optimization (ICOMP 2025)
Algorithms:
R-LSVRG, R-PAGE, R-MARINA
[244] Laurent Condat, Artavazd Maranjyan, and Peter Richtárik
LoCoDL: Communication-efficient distributed learning with local training and compression
13th International Conference on Learning Representations (ICLR 2025)
NeurIPS 2024 Workshop: Optimization for Machine Learning (OPT 2024)
Spotlight at ICLR 2025
arXiv poster
Algorithms:
LoCoDL: Communication-efficient distributed learning with local training and compression
13th International Conference on Learning Representations (ICLR 2025)
NeurIPS 2024 Workshop: Optimization for Machine Learning (OPT 2024)
Spotlight at ICLR 2025
arXiv poster
Algorithms:
LoCoDL
[243] Kaja Gruntkowska, Alexander Tyurin, and Peter Richtárik
Improving the worst-case bidirectional communication complexity for nonconvex distributed optimization under function similarity
Advances in Neural Information Processing Systems 38 (NeurIPS 2024)
Spotlight at NeurIPS 2024
arXiv poster
Algorithms:
Improving the worst-case bidirectional communication complexity for nonconvex distributed optimization under function similarity
Advances in Neural Information Processing Systems 38 (NeurIPS 2024)
Spotlight at NeurIPS 2024
arXiv poster
Algorithms:
MARINA-P, M3
[242] Alexander Tyurin, Marta Pozzi, Ivan Ilin, and Peter Richtárik
Shadowheart SGD: Distributed asynchronous SGD with optimal time complexity under arbitrary computation and communication heterogeneity
Advances in Neural Information Processing Systems 38 (NeurIPS 2024)
arXiv poster slides
Algorithms:
Shadowheart SGD: Distributed asynchronous SGD with optimal time complexity under arbitrary computation and communication heterogeneity
Advances in Neural Information Processing Systems 38 (NeurIPS 2024)
arXiv poster slides
Algorithms:
Shadowheart SGD
[241] Andrei Panferov, Yury Demidovich, Ahmad Rammal, and Peter Richtárik
Correlated quantization for faster nonconvex distributed optimization
41st Conference on Uncertainty in Artificial Intelligence (UAI 2025)
Oral at UAI 2025
arXiv
Algorithms:
Correlated quantization for faster nonconvex distributed optimization
41st Conference on Uncertainty in Artificial Intelligence (UAI 2025)
Oral at UAI 2025
arXiv
Algorithms:
MARINA, PermK+CQPrepared in 2023
[240] Kai Yi, Nidham Gazagnadou, Peter Richtárik, and Lingjuan Lyu
FedP3: Personalized and privacy-friendly federated network pruning under model heterogeneity
12th International Conference on Learning Representations (ICLR 2024)
arXiv poster
Algorithms:
FedP3: Personalized and privacy-friendly federated network pruning under model heterogeneity
12th International Conference on Learning Representations (ICLR 2024)
arXiv poster
Algorithms:
FedP3
[239] Peter Richtárik, Elnur Gasanov, Konstantin Burlachenko
Error feedback reloaded: From quadratic to arithmetic mean of smoothness constants
12th International Conference on Learning Representations (ICLR 2024)
arXiv
Algorithms:
Error feedback reloaded: From quadratic to arithmetic mean of smoothness constants
12th International Conference on Learning Representations (ICLR 2024)
arXiv
Algorithms:
EF21-W, EF21
[238] Jihao Xin, Ivan Ilin, Shunkang Zhang, Marco Canini, Peter Richtárik
Kimad: Adaptive gradient compression with bandwidth awareness
Proceedings of the 4th International Workshop on Distributed Machine Learning, 25--48, 2023 (DistributedML 2023)
arXiv
Algorithms:
Kimad: Adaptive gradient compression with bandwidth awareness
Proceedings of the 4th International Workshop on Distributed Machine Learning, 25--48, 2023 (DistributedML 2023)
arXiv
Algorithms:
Kimad, Kimad+
[237] Konstantin Burlachenko, Abdulmajeed Alrowithi, Fahad Ali Albalawi, and Peter Richtárik
Federated learning is better with non-homomorphic encryption
Proceedings of the 4th International Workshop on Distributed Machine Learning, 49--84, 2023 (DistributedML 2023)
arXiv
Algorithms:
Federated learning is better with non-homomorphic encryption
Proceedings of the 4th International Workshop on Distributed Machine Learning, 49--84, 2023 (DistributedML 2023)
arXiv
Algorithms:
DCGD/PermK/AES
[236] Yury Demidovich, Grigory Malinovsky, Egor Shulgin, and Peter Richtárik
MAST: model-agnostic sparsified training
13th International Conference on Learning Representations (ICLR 2025)
arXiv
Algorithms:
MAST: model-agnostic sparsified training
13th International Conference on Learning Representations (ICLR 2025)
arXiv
Algorithms:
double sketched (S)GD, distributed double sketched GD, L-SVRDSG, S-PAGE
[235] Grigory Malinovsky, Peter Richtárik, Samuel Horváth, and Eduard Gorbunov
Byzantine robustness and partial participation can be achieved simultaneously: just clip gradient differences
Advances in Neural Information Processing Systems 38 (NeurIPS 2024)
arXiv poster
Algorithms:
Byzantine robustness and partial participation can be achieved simultaneously: just clip gradient differences
Advances in Neural Information Processing Systems 38 (NeurIPS 2024)
arXiv poster
Algorithms:
Byz-VR-MARINA-PP
[234] Massimo Fornasier, Peter Richtárik, Konstantin Riedl, and Lukang Sun
Consensus-based optimization with truncated noise
arXiv
Algorithms:
Consensus-based optimization with truncated noise
arXiv
Algorithms:
CBO
[233] Ahmad Rammal, Kaja Gruntkowska, Nikita Fedin, Eduard Gorbunov, and Peter Richtárik
Communication compression for Byzantine robust learning: New efficient algorithms and improved rates
26th International Conference on Artificial Intelligence and Statistics (AISTATS 2024)
arXiv poster
Algorithms:
Communication compression for Byzantine robust learning: New efficient algorithms and improved rates
26th International Conference on Artificial Intelligence and Statistics (AISTATS 2024)
arXiv poster
Algorithms:
Byz-VR-MARINA, Byz-DASHA-PAGE, Byz-EF21, Byz-EF21-BC
[232] Hanmin Li, Avetik Karagulyan, and Peter Richtárik
MARINA meets matrix stepsizes: Variance reduced distributed non-convex optimization
arXiv
Algorithms:
MARINA meets matrix stepsizes: Variance reduced distributed non-convex optimization
arXiv
Algorithms:
det-MARINA
[231] Eduard Gorbunov, Abdurakhmon Sadiev, Marina Danilova, Samuel Horváth, Gauthier Gidel, Pavel Dvurechensky, Alexander Gasnikov, and Peter Richtárik
High-probability convergence for composite and distributed stochastic minimization and variational inequalities with heavy-tailed noise
41st International Conference on Machine Learning (ICML 2024)
Oral (144/9473 = top 1.5\%)
arXiv poster
Algorithms:
High-probability convergence for composite and distributed stochastic minimization and variational inequalities with heavy-tailed noise
41st International Conference on Machine Learning (ICML 2024)
Oral (144/9473 = top 1.5\%)
arXiv poster
Algorithms:
DProx-clipped-SGD-shift, DProx-clipped-SSTM-shift
[230] Egor Shulgin and Peter Richtárik
Towards a better theoretical understanding of independent subnetwork training
41st International Conference on Machine Learning (ICML 2024)
Federated Learning and Analytics in Practice: Algorithms, Systems, Applications, and Opportunities (ICML 2023 Workshop)
arXiv poster
Algorithms:
Towards a better theoretical understanding of independent subnetwork training
41st International Conference on Machine Learning (ICML 2024)
Federated Learning and Analytics in Practice: Algorithms, Systems, Applications, and Opportunities (ICML 2023 Workshop)
arXiv poster
Algorithms:
IST
[229] Rafał Szlendak, Elnur Gasanov, and Peter Richtárik
Understanding progressive training through the framework of randomized coordinate descent
26th International Conference on Artificial Intelligence and Statistics (AISTATS 2024)
arXiv
Algorithms:
Understanding progressive training through the framework of randomized coordinate descent
26th International Conference on Artificial Intelligence and Statistics (AISTATS 2024)
arXiv
Algorithms:
RPT
[228] Michał Grudzień, Grigory Malinovsky, and Peter Richtárik
Improving accelerated federated learning with compression and importance sampling
Federated Learning and Analytics in Practice: Algorithms, Systems, Applications, and Opportunities (ICML 2023 Workshop)
arXiv
Algorithms:
Improving accelerated federated learning with compression and importance sampling
Federated Learning and Analytics in Practice: Algorithms, Systems, Applications, and Opportunities (ICML 2023 Workshop)
arXiv
Algorithms:
5GCS-CC, 5GCS-AB
[227] Sarit Khirirat, Eduard Gorbunov, Samuel Horváth, Rustem Islamov, Fakhri Karray, and Peter Richtárik
Clip21: Error feedback for gradient clipping
arXiv
Algorithms:
Clip21: Error feedback for gradient clipping
arXiv
Algorithms:
Clip21-Avg, Clip21-GD, DP-Clip21-GD, Press-Clip21-GD
[226] Jihao Xin, Marco Canini, Peter Richtárik, and Samuel Horváth
Quantize once, train fast: allreduce-compatible compression with provable guarantees
28th European Conference on Artificial Intelligence (ECAI 2025)
arXiv
Algorithms:
Quantize once, train fast: allreduce-compatible compression with provable guarantees
28th European Conference on Artificial Intelligence (ECAI 2025)
arXiv
Algorithms:
Global-QSGD
[225] Yury Demidovich, Grigory Malinovsky, Igor Sokolov and Peter Richtárik
A guide through the zoo of biased SGD
Advances in Neural Information Processing Systems 36 (NeurIPS 2023)
arXiv poster
Algorithms:
A guide through the zoo of biased SGD
Advances in Neural Information Processing Systems 36 (NeurIPS 2023)
arXiv poster
Algorithms:
BiasedSGD
[224] Peter Richtárik, Elnur Gasanov and Konstantin Burlachenko
Error feedback shines when features are rare
arXiv
Algorithms:
Error feedback shines when features are rare
arXiv
Algorithms:
EF21
[223] Ilyas Fatkhullin, Alexander Tyurin and Peter Richtárik
Momentum provably improves error feedback!
Advances in Neural Information Processing Systems 36 (NeurIPS 2023)
Federated Learning and Analytics in Practice: Algorithms, Systems, Applications, and Opportunities (ICML 2023 Workshop)
arXiv poster
Algorithms:
Momentum provably improves error feedback!
Advances in Neural Information Processing Systems 36 (NeurIPS 2023)
Federated Learning and Analytics in Practice: Algorithms, Systems, Applications, and Opportunities (ICML 2023 Workshop)
arXiv poster
Algorithms:
EF21-SGDM-ideal, EF21-SGDM, EF21-SGD2M
[222] Kai Yi, Laurent Condat and Peter Richtárik
Explicit personalization and local training: double communication acceleration in federated learning
Transactions on Machine Learning Research (TMLR 2025)
arXiv
Algorithms:
Explicit personalization and local training: double communication acceleration in federated learning
Transactions on Machine Learning Research (TMLR 2025)
arXiv
Algorithms:
Scafflix
[221] Alexander Tyurin and Peter Richtárik
Optimal time complexities of parallel stochastic optimization methods under a fixed computation model
Advances in Neural Information Processing Systems 36 (NeurIPS 2023)
arXiv video poster slides
Algorithms:
Optimal time complexities of parallel stochastic optimization methods under a fixed computation model
Advances in Neural Information Processing Systems 36 (NeurIPS 2023)
arXiv video poster slides
Algorithms:
Rennala SGD, Malenia SGD
[220] Alexander Tyurin and Peter Richtárik
2Direction: Theoretically faster distributed training with bidirectional communication compression
Advances in Neural Information Processing Systems 36 (NeurIPS 2023)
arXiv poster
Algorithms:
2Direction: Theoretically faster distributed training with bidirectional communication compression
Advances in Neural Information Processing Systems 36 (NeurIPS 2023)
arXiv poster
Algorithms:
2Direction
[219] Hanmin Li, Avetik Karagulyan and Peter Richtárik
Det-CGD: Compressed gradient descent with matrix stepsizes for non-convex optimization
12th International Conference on Learning Representations (ICLR 2024)
arXiv poster
Algorithms:
Det-CGD: Compressed gradient descent with matrix stepsizes for non-convex optimization
12th International Conference on Learning Representations (ICLR 2024)
arXiv poster
Algorithms:
Det-CGD
[218] Avetik Karagulyan and Peter Richtárik
ELF: Federated Langevin algorithms with primal, dual and bidirectional compression
41st Conference on Uncertainty in Artificial Intelligence (UAI 2025)
Federated Learning and Analytics in Practice: Algorithms, Systems, Applications, and Opportunities (ICML 2023 Workshop)
arXiv
Algorithms:
ELF: Federated Langevin algorithms with primal, dual and bidirectional compression
41st Conference on Uncertainty in Artificial Intelligence (UAI 2025)
Federated Learning and Analytics in Practice: Algorithms, Systems, Applications, and Opportunities (ICML 2023 Workshop)
arXiv
Algorithms:
ELF, P-ELF, D-ELF, B-ELF
[217] Laurent Condat, Grigory Malinovsky and Peter Richtárik
TAMUNA: Doubly accelerated distributed optimization under partial participation
arXiv
Algorithms:
TAMUNA: Doubly accelerated distributed optimization under partial participation
arXiv
Algorithms:
TAMUNA
[216] Grigory Malinovsky, Samuel Horváth, Konstantin Burlachenko and Peter Richtárik
Federated learning with regularized client participation
Federated Learning and Analytics in Practice: Algorithms, Systems, Applications, and Opportunities (ICML 2023 Workshop)
arXiv
Algorithms:
Federated learning with regularized client participation
Federated Learning and Analytics in Practice: Algorithms, Systems, Applications, and Opportunities (ICML 2023 Workshop)
arXiv
Algorithms:
RR-CLI
[215] Abdurakhmon Sadiev, Marina Danilova, Eduard Gorbunov, Samuel Horváth, Gauthier Gidel, Pavel Dvurechensky, Alexander Gasnikov and Peter Richtárik
High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance
40th International Conference on Machine Learning (ICML 2023)
arXiv poster
Algorithms:
High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance
40th International Conference on Machine Learning (ICML 2023)
arXiv poster
Algorithms:
clipped-SGD, clipped-SSTM, R-clipped-SSTM
[214] Xun Qian, Hanze Dong, Tong Zhang and Peter Richtárik
Catalyst acceleration of error compensated methods leads to better communication complexity
25th International Conference on Artificial Intelligence and Statistics (AISTATS 2023)
arXiv poster
Algorithms:
Catalyst acceleration of error compensated methods leads to better communication complexity
25th International Conference on Artificial Intelligence and Statistics (AISTATS 2023)
arXiv poster
Algorithms:
ECSPDC, EC-LSVRG + Catalyst, EC-SDCA + Catalyst
[213] Slavomír Hanzely, Konstantin Mishchenko and Peter Richtárik
Convergence of first-order algorithms for meta-learning with Moreau envelopes
Federated Learning and Analytics in Practice: Algorithms, Systems, Applications, and Opportunities (ICML 2023 Workshop)
arXiv
Algorithms:
Convergence of first-order algorithms for meta-learning with Moreau envelopes
Federated Learning and Analytics in Practice: Algorithms, Systems, Applications, and Opportunities (ICML 2023 Workshop)
arXiv
Algorithms:
FO-MuML
Prepared in 2022
[212] Michał Grudzień, Grigory Malinovsky and Peter Richtárik
Can 5th generation local training methods support client sampling? Yes!
25th International Conference on Artificial Intelligence and Statistics (AISTATS 2023)
arXiv poster
Algorithms:
Can 5th generation local training methods support client sampling? Yes!
25th International Conference on Artificial Intelligence and Statistics (AISTATS 2023)
arXiv poster
Algorithms:
5GCS
[211] Maksim Makarenko, Elnur Gasanov, Rustem Islamov, Abdurakhmon Sadiev and Peter Richtárik
Adaptive compression for communication-efficient distributed training
Transactions on Machine Learning Research (TMLR 2023)
arXiv
Algorithms:
Adaptive compression for communication-efficient distributed training
Transactions on Machine Learning Research (TMLR 2023)
arXiv
Algorithms:
AdaCGD
[210] Slavomír Hanzely, Dmitry Kamzolov, Dmitry Pasechnyuk, Alexander Gasnikov, Peter Richtárik and Martin Takáč
A damped Newton method achieves global $O (1/k^2)$ and local quadratic convergence rate
Advances in Neural Information Processing Systems 35 (NeurIPS 2022)
arXiv poster
Algorithms:
A damped Newton method achieves global $O (1/k^2)$ and local quadratic convergence rate
Advances in Neural Information Processing Systems 35 (NeurIPS 2022)
arXiv poster
Algorithms:
AIC Newton
[209] Artavazd Maranjyan, Mher Safaryan and Peter Richtárik
GradSkip: Communication-accelerated local gradient methods with better computational complexity
arXiv
Algorithms:
GradSkip: Communication-accelerated local gradient methods with better computational complexity
arXiv
Algorithms:
GradSkip, GradSkip+
[208] Laurent Condat, Ivan Agarský and Peter Richtárik
CompressedScaffnew: the first theoretical double acceleration of communication from local training and compression in distributed optimization
Optimization, 2026
arXiv
Algorithms:
CompressedScaffnew: the first theoretical double acceleration of communication from local training and compression in distributed optimization
Optimization, 2026
arXiv
Algorithms:
CompressedScaffnew
[207] Lukang Sun and Peter Richtárik
Improved Stein variational gradient descent with importance weights
arXiv
Algorithms:
Improved Stein variational gradient descent with importance weights
arXiv
Algorithms:
beta-SVGD
[206] Kaja Gruntkowska, Alexander Tyurin and Peter Richtárik
EF21-P and friends: Improved theoretical communication complexity for distributed optimization with bidirectional compression
40th International Conference on Machine Learning (ICML 2023)
arXiv poster
Algorithms:
EF21-P and friends: Improved theoretical communication complexity for distributed optimization with bidirectional compression
40th International Conference on Machine Learning (ICML 2023)
arXiv poster
Algorithms:
EF21-P, EF21-P + DIANA, EF21-P + DCGD
[205] Soumia Boucherouite, Grigory Malinovsky, Peter Richtárik and El Houcine Bergou
Minibatch stochastic three points method for unconstrained smooth minimization
38th AAAI Conference on Artificial Intelligence (AAAI 2024)
arXiv
Algorithms:
Minibatch stochastic three points method for unconstrained smooth minimization
38th AAAI Conference on Artificial Intelligence (AAAI 2024)
arXiv
Algorithms:
MiSTP
[204] El Houcine Bergou, Konstantin Burlachenko, Aritra Dutta and Peter Richtárik
Personalized federated learning with communication compression
Transactions on Machine Learning Research (TMLR 2023)
arXiv github
Algorithms:
Personalized federated learning with communication compression
Transactions on Machine Learning Research (TMLR 2023)
arXiv github
Algorithms:
Compressed L2GD
[203] Samuel Horváth, Konstantin Mishchenko and Peter Richtárik
Adaptive learning rates for faster stochastic gradient methods
arXiv
Algorithms:
Adaptive learning rates for faster stochastic gradient methods
arXiv
Algorithms:
StoPS, GraDs, StoP, GraD
[202] Laurent Condat and Peter Richtárik
RandProx: Primal-dual optimization algorithms with randomized proximal updates
11th International Conference on Learning Representations (ICLR 2023)
OPT2022: 14th Annual Workshop on Optimization for Machine Learning (NeurIPS 2022 Workshop)
arXiv video poster
Algorithms:
RandProx: Primal-dual optimization algorithms with randomized proximal updates
11th International Conference on Learning Representations (ICLR 2023)
OPT2022: 14th Annual Workshop on Optimization for Machine Learning (NeurIPS 2022 Workshop)
arXiv video poster
Algorithms:
RandProx, RandProx-FB, RandProx-LC, RandProx-CP, RandProx-ADMM, RandProx-DY
[201] Grigory Malinovsky, Kai Yi and Peter Richtárik
Variance reduced ProxSkip: algorithm, theory and application to federated learning
Advances in Neural Information Processing Systems 35 (NeurIPS 2022)
arXiv
Algorithms:
Variance reduced ProxSkip: algorithm, theory and application to federated learning
Advances in Neural Information Processing Systems 35 (NeurIPS 2022)
arXiv
Algorithms:
ProxSkip-VR, ProxSkip-GD, ProxSkip-SGD, ProxSkip-LSVRG, ProxSkip-HUB
[200] Abdurakhmon Sadiev, Dmitry Kovalev and Peter Richtárik
Communication acceleration of local gradient methods via an accelerated primal-dual algorithm with inexact prox
Advances in Neural Information Processing Systems 35 (NeurIPS 2022)
arXiv
Algorithms:
Communication acceleration of local gradient methods via an accelerated primal-dual algorithm with inexact prox
Advances in Neural Information Processing Systems 35 (NeurIPS 2022)
arXiv
Algorithms:
APDA, APDA with Inexact Prox, APDA with Inexact Prox and Accelerated Gossip
[199] Egor Shulgin and Peter Richtárik
Shifted compression framework: generalizations and improvements
38th Conference on Uncertainty in Artificial Intelligence (UAI 2022)
arXiv poster
Algorithms:
Shifted compression framework: generalizations and improvements
38th Conference on Uncertainty in Artificial Intelligence (UAI 2022)
arXiv poster
Algorithms:
DCGD-SHIFT
[198] Lukang Sun and Peter Richtárik
A Note on the convergence of mirrored Stein variational gradient descent under (L_0, L_1) smoothness condition
arXiv
Algorithms:
A Note on the convergence of mirrored Stein variational gradient descent under (L_0, L_1) smoothness condition
arXiv
Algorithms:
MSVGD
[197] Abdurakhmon Sadiev, Grigory Malinovsky, Eduard Gorbunov, Igor Sokolov, Ahmed Khaled, Konstantin Burlachenko and Peter Richtárik
Don't compress gradients in random reshuffling: compress gradient differences
Advances in Neural Information Processing Systems 38 (NeurIPS 2024)
Federated Learning and Analytics in Practice: Algorithms, Systems, Applications, and Opportunities (ICML 2023 Workshop)
arXiv poster
Algorithms:
Don't compress gradients in random reshuffling: compress gradient differences
Advances in Neural Information Processing Systems 38 (NeurIPS 2024)
Federated Learning and Analytics in Practice: Algorithms, Systems, Applications, and Opportunities (ICML 2023 Workshop)
arXiv poster
Algorithms:
Q-RR, DIANA-RR, Q-NASTYA, DIANA-NASTYA
[196] Rustem Islamov, Xun Qian, Slavomír Hanzely, Mher Safaryan and Peter Richtárik
Distributed Newton-type methods with communication compression and Bernoulli aggregation
Transactions on Machine Learning Research (TMLR 2023)
NeurIPS Workshop 2022 (Order up! The Benefits of Higher-Order Optimization in Machine Learning)
arXiv
Algorithms:
Distributed Newton-type methods with communication compression and Bernoulli aggregation
Transactions on Machine Learning Research (TMLR 2023)
NeurIPS Workshop 2022 (Order up! The Benefits of Higher-Order Optimization in Machine Learning)
arXiv
Algorithms:
Newton-3PC, Newton-3PC-BC, Newton-3PC-BC-PP
[195] Motasem Alfarra, Juan C. Pérez, Egor Shulgin, Peter Richtárik and Bernard Ghanem
Certified robustness in federated learning
NeurIPS Workshop 2022 (Federated Learning)
arXiv
Certified robustness in federated learning
NeurIPS Workshop 2022 (Federated Learning)
arXiv
[194] Alexander Tyurin, Lukang Sun, Konstantin Burlachenko and Peter Richtárik
Sharper rates and flexible framework for nonconvex SGD with client and data sampling
Transactions on Machine Learning Research (TMLR 2023)
arXiv
Algorithms:
Sharper rates and flexible framework for nonconvex SGD with client and data sampling
Transactions on Machine Learning Research (TMLR 2023)
arXiv
Algorithms:
PAGE
[193] Lukang Sun, Adil Salim and Peter Richtárik
Federated sampling with Langevin algorithm under isoperimetry
Transactions on Machine Learning Research (TMLR 2024)
arXiv
Algorithms:
Federated sampling with Langevin algorithm under isoperimetry
Transactions on Machine Learning Research (TMLR 2024)
arXiv
Algorithms:
Langevin-Marina
[192] Eduard Gorbunov, Samuel Horváth, Peter Richtárik and Gauthier Gidel
Variance reduction is an antidote to Byzantines: better rates, weaker assumptions and communication compression as a cherry on the top
11th International Conference on Learning Representations (ICLR 2023)
arXiv poster
Algorithms:
Variance reduction is an antidote to Byzantines: better rates, weaker assumptions and communication compression as a cherry on the top
11th International Conference on Learning Representations (ICLR 2023)
arXiv poster
Algorithms:
Byz-VR-MARINA
[191] Lukang Sun, Avetik Karagulyan and Peter Richtárik
Convergence of Stein variational gradient descent under a weaker smoothness condition
25th International Conference on Artificial Intelligence and Statistics (AISTATS 2023)
arXiv poster
Algorithms:
Convergence of Stein variational gradient descent under a weaker smoothness condition
25th International Conference on Artificial Intelligence and Statistics (AISTATS 2023)
arXiv poster
Algorithms:
SVGD
[190] Alexander Tyurin and Peter Richtárik
A computation and communication efficient method for distributed nonconvex problems in the partial participation setting
Advances in Neural Information Processing Systems 36 (NeurIPS 2023)
arXiv poster
Algorithms:
A computation and communication efficient method for distributed nonconvex problems in the partial participation setting
Advances in Neural Information Processing Systems 36 (NeurIPS 2023)
arXiv poster
Algorithms:
DASHA-PP, DASHA-PP-PAGE, DASHA-PP-FINITE-MVR, DASHA-PP-MVR
[189] Laurent Condat, Kai Yi and Peter Richtárik
EF-BV: A unified theory of error feedback and variance reduction mechanisms for biased and unbiased compression in distributed optimization
Advances in Neural Information Processing Systems 35 (NeurIPS 2022)
arXiv poster
Algorithms:
EF-BV: A unified theory of error feedback and variance reduction mechanisms for biased and unbiased compression in distributed optimization
Advances in Neural Information Processing Systems 35 (NeurIPS 2022)
arXiv poster
Algorithms:
EF-BV
[188] Grigory Malinovsky and Peter Richtárik
Federated random reshuffling with compression and variance reduction
arXiv
Algorithms:
Federated random reshuffling with compression and variance reduction
arXiv
Algorithms:
FedCRR, FedCRR-VR, FedCRR-VR-2
[187] Samuel Horváth, Maziar Sanjabi, Lin Xiao, Peter Richtárik and Michael Rabbat
FedShuffle: Recipes for better use of local work in federated learning
Transactions on Machine Learning Research (TMLR 2022)
arXiv
Algorithms:
FedShuffle: Recipes for better use of local work in federated learning
Transactions on Machine Learning Research (TMLR 2022)
arXiv
Algorithms:
FedShuffle
[186] Konstantin Mishchenko, Grigory Malinovsky, Sebastian Stich and Peter Richtárik
ProxSkip: Yes! Local gradient steps provably lead to communication acceleration! Finally!
39th International Conference on Machine Learning (ICML 2022)
arXiv slides video
Algorithms:
ProxSkip: Yes! Local gradient steps provably lead to communication acceleration! Finally!
39th International Conference on Machine Learning (ICML 2022)
arXiv slides video
Algorithms:
ProxSkip, Scaffnew, SProxSkip, SplitSkip, Decentralized Scaffnew
[185] Dmitry Kovalev, Aleksandr Beznosikov, Abdurakhmon Sadiev, Michael Persiianov, Peter Richtárik and Alexander Gasnikov
Optimal algorithms for decentralized stochastic variational inequalities
Advances in Neural Information Processing Systems 35 (NeurIPS 2022)
arXiv
Algorithms:
Optimal algorithms for decentralized stochastic variational inequalities
Advances in Neural Information Processing Systems 35 (NeurIPS 2022)
arXiv
Algorithms:
Algorithm 1, Algorithm 2
[184] Alexander Tyurin and Peter Richtárik
DASHA: Distributed nonconvex optimization with communication compression and optimal oracle complexity
10th International Conference on Learning Representations (ICLR 2023)
Oral Paper at ICLR 2023
arXiv poster
Algorithms:
DASHA: Distributed nonconvex optimization with communication compression and optimal oracle complexity
10th International Conference on Learning Representations (ICLR 2023)
Oral Paper at ICLR 2023
arXiv poster
Algorithms:
DASHA, DASHA-PAGE, DASHA-MVR
[183] Peter Richtárik, Igor Sokolov, Ilyas Fatkhullin, Elnur Gasanov, Zhize Li and Eduard Gorbunov
3PC: Three point compressors for communication-efficient distributed training and a better theory for lazy aggregation
39th International Conference on Machine Learning (ICML 2022)
arXiv poster
Algorithms:
3PC: Three point compressors for communication-efficient distributed training and a better theory for lazy aggregation
39th International Conference on Machine Learning (ICML 2022)
arXiv poster
Algorithms:
3PC, LAG, CLAG, EF21
[182] Haoyu Zhao, Boyue Li, Zhize Li, Peter Richtárik and Yuejie Chi
BEER: Fast $O (1/T)$ rate for decentralized nonconvex optimization with communication compression
Advances in Neural Information Processing Systems 35 (NeurIPS 2022)
arXiv poster
Algorithms:
BEER: Fast $O (1/T)$ rate for decentralized nonconvex optimization with communication compression
Advances in Neural Information Processing Systems 35 (NeurIPS 2022)
arXiv poster
Algorithms:
BEER
[181] Grigory Malinovsky, Konstantin Mishchenko and Peter Richtárik
Server-side stepsizes and sampling without replacement provably help in federated optimization
Proceedings of the 4th International Workshop on Distributed Machine Learning, 85--104, 2023 (DistributedML 2023) arXiv
Algorithms:
Server-side stepsizes and sampling without replacement provably help in federated optimization
Proceedings of the 4th International Workshop on Distributed Machine Learning, 85--104, 2023 (DistributedML 2023) arXiv
Algorithms:
Nastya
Prepared in 2021
[180] Dmitry Kovalev, Alexander Gasnikov and Peter Richtárik
Accelerated primal-dual gradient method for smooth and convex-concave saddle-point problems with bilinear coupling
Advances in Neural Information Processing Systems 35 (NeurIPS 2022)
arXiv
Algorithms:
Accelerated primal-dual gradient method for smooth and convex-concave saddle-point problems with bilinear coupling
Advances in Neural Information Processing Systems 35 (NeurIPS 2022)
arXiv
Algorithms:
APDG
[179] Haoyu Zhao, Konstantin Burlachenko, Zhize Li and Peter Richtárik
Faster rates for compressed federated learning with client-variance reduction
SIAM Journal on Mathematics of Data Science 6 (1):154-175, 2024
arXiv
Algorithms:
Faster rates for compressed federated learning with client-variance reduction
SIAM Journal on Mathematics of Data Science 6 (1):154-175, 2024
arXiv
Algorithms:
COFIG, FRECON
[178] Konstantin Burlachenko, Samuel Horváth and Peter Richtárik
FL_PyTorch: optimization research simulator for federated learning
Proceedings of the 2nd ACM International Workshop on Distributed Machine Learning
arXiv
Algorithms:
FL_PyTorch: optimization research simulator for federated learning
Proceedings of the 2nd ACM International Workshop on Distributed Machine Learning
arXiv
Algorithms:
FL_PyTorch
[177] Elnur Gasanov, Ahmed Khaled, Samuel Horváth and Peter Richtárik
FLIX: A simple and communication-efficient alternative to local methods in federated learning
24th International Conference on Artificial Intelligence and Statistics (AISTATS 2022)
arXiv poster
Algorithms:
FLIX: A simple and communication-efficient alternative to local methods in federated learning
24th International Conference on Artificial Intelligence and Statistics (AISTATS 2022)
arXiv poster
Algorithms:
FLIX
[176] Xun Qian, Rustem Islamov, Mher Safaryan and Peter Richtárik
Basis matters: better communication-efficient second order methods for federated learning
24th International Conference on Artificial Intelligence and Statistics (AISTATS 2022)
arXiv poster
Algorithms:
Basis matters: better communication-efficient second order methods for federated learning
24th International Conference on Artificial Intelligence and Statistics (AISTATS 2022)
arXiv poster
Algorithms:
BL1, BL2, BL3
[175] Aleksandr Beznosikov, Peter Richtárik, Michael Diskin, Max Ryabinin and Alexander Gasnikov
Distributed methods with compressed communication for solving variational inequalities, with theoretical guarantees
Advances in Neural Information Processing Systems 35 (NeurIPS 2022)
arXiv
Algorithms:
Distributed methods with compressed communication for solving variational inequalities, with theoretical guarantees
Advances in Neural Information Processing Systems 35 (NeurIPS 2022)
arXiv
Algorithms:
MASHA1, MASHA2
[174] Rafał Szlendak, Alexander Tyurin and Peter Richtárik
Permutation compressors for provably faster distributed nonconvex optimization
10th International Conference on Learning Representations (ICLR 2022)
arXiv video 1 video 2 video 3 poster
Algorithms:
Permutation compressors for provably faster distributed nonconvex optimization
10th International Conference on Learning Representations (ICLR 2022)
arXiv video 1 video 2 video 3 poster
Algorithms:
MARINA
[173] Ilyas Fatkhullin, Igor Sokolov, Eduard Gorbunov, Zhize Li and Peter Richtárik
EF21 with bells & whistles: practical algorithmic extensions of modern error feedback
Journal of Machine Learning Research, 2025
arXiv github
Algorithms:
EF21 with bells & whistles: practical algorithmic extensions of modern error feedback
Journal of Machine Learning Research, 2025
arXiv github
Algorithms:
EF21-SGD, EF21-PAGE, EF21-PP, EF21-BC, EF21-HB, EF21-Prox
[172] Xun Qian, Hanze Dong, Peter Richtárik and Tong Zhang
Error compensated loopless SVRG, Quartz, and SDCA for distributed optimization
arXiv
Algorithms:
Error compensated loopless SVRG, Quartz, and SDCA for distributed optimization
arXiv
Algorithms:
EC-LSVRG, EC-SDCA, EC-Quartz
[171] Majid Jahani, Sergey Rusakov, Zheng Shi, Peter Richtárik, Michael W. Mahoney and Martin Takáč
Doubly adaptive scaled algorithm for machine learning using second-order information
10th International Conference on Learning Representations (ICLR 2022)
arXiv poster
Algorithms:
Doubly adaptive scaled algorithm for machine learning using second-order information
10th International Conference on Learning Representations (ICLR 2022)
arXiv poster
Algorithms:
OASIS
[170] Haoyu Zhao, Zhize Li and Peter Richtárik
FedPAGE: A fast local stochastic gradient method for communication-efficient federated learning
arXiv
Algorithms:
FedPAGE: A fast local stochastic gradient method for communication-efficient federated learning
arXiv
Algorithms:
FedPAGE
[169] Zhize Li and Peter Richtárik
CANITA: Faster rates for distributed convex optimization with communication compression
Advances in Neural Information Processing Systems 34 (NeurIPS 2021)
arXiv poster
Algorithms:
CANITA: Faster rates for distributed convex optimization with communication compression
Advances in Neural Information Processing Systems 34 (NeurIPS 2021)
arXiv poster
Algorithms:
CANITA
[167] Peter Richtárik, Igor Sokolov and Ilyas Fatkhullin
EF21: A new, simpler, theoretically better, and practically faster error feedback
Advances in Neural Information Processing Systems 34 (NeurIPS 2021)
NeurIPS 2021 oral paper (less than 1\% acceptance rate)
arXiv slides video 1 video 2 poster github
Algorithms:
EF21: A new, simpler, theoretically better, and practically faster error feedback
Advances in Neural Information Processing Systems 34 (NeurIPS 2021)
NeurIPS 2021 oral paper (less than 1\% acceptance rate)
arXiv slides video 1 video 2 poster github
Algorithms:
EF21, EF21+
[166] Dmitry Kovalev, Elnur Gasanov, Peter Richtárik and Alexander Gasnikov
Lower bounds and optimal algorithms for smooth and strongly convex decentralized optimization over time-varying networks
Advances in Neural Information Processing Systems 34 (NeurIPS 2021)
arXiv poster
Algorithms:
Lower bounds and optimal algorithms for smooth and strongly convex decentralized optimization over time-varying networks
Advances in Neural Information Processing Systems 34 (NeurIPS 2021)
arXiv poster
Algorithms:
ADOM+
[165] Bokun Wang, Mher Safaryan and Peter Richtárik
Theoretically better and numerically faster distributed optimization with smoothness-aware quantization techniques
Advances in Neural Information Processing Systems 35 (NeurIPS 2022)
arXiv poster
Algorithms:
Theoretically better and numerically faster distributed optimization with smoothness-aware quantization techniques
Advances in Neural Information Processing Systems 35 (NeurIPS 2022)
arXiv poster
Algorithms:
DCGD+, DIANA+
[164] Adil Salim, Lukang Sun and Peter Richtárik
A convergence theory for SVGD in the population limit under Talagrand’s inequality T1
39th International Conference on Machine Learning (ICML 2022)
arXiv
Algorithms:
A convergence theory for SVGD in the population limit under Talagrand’s inequality T1
39th International Conference on Machine Learning (ICML 2022)
arXiv
Algorithms:
SVGD
[163] Laurent Condat and Peter Richtárik
MURANA: A generic framework for stochastic variance-reduced optimization
Mathematical and Scientific Machine Learning 2022 (MSML 2022)
arXiv
Algorithms:
MURANA: A generic framework for stochastic variance-reduced optimization
Mathematical and Scientific Machine Learning 2022 (MSML 2022)
arXiv
Algorithms:
MURANA, ELVIRA
[162] Mher Safaryan, Rustem Islamov, Xun Qian and Peter Richtárik
FedNL: Making Newton-type methods applicable to federated learning
39th International Conference on Machine Learning (ICML 2022)
arXiv poster
Algorithms:
FedNL: Making Newton-type methods applicable to federated learning
39th International Conference on Machine Learning (ICML 2022)
arXiv poster
Algorithms:
FedNL, FedNL-PP, FedNL-CR, FedNL-LS, FedNL-BC, N0, NS
[161] Grigory Malinovsky, Alibek Sailanbayev and Peter Richtárik
Random reshuffling with variance reduction: new analysis and better rates
39th Conference on Uncertainty in Artificial Intelligence (UAI 2023)
arXiv video
Algorithms:
Random reshuffling with variance reduction: new analysis and better rates
39th Conference on Uncertainty in Artificial Intelligence (UAI 2023)
arXiv video
Algorithms:
RR-SVRG, SO-SVRG, Cyclic-SVRG
[160] Zhize Li, Slavomír Hanzely and Peter Richtárik
ZeroSARAH: Efficient nonconvex finite-sum optimization with zero full gradient computation
arXiv
Algorithms:
ZeroSARAH: Efficient nonconvex finite-sum optimization with zero full gradient computation
arXiv
Algorithms:
Zero-SARAH
[159] Adil Salim, Laurent Condat, Dmitry Kovalev and Peter Richtárik
An optimal algorithm for strongly convex minimization under affine constraints
24th International Conference on Artificial Intelligence and Statistics (AISTATS 2022)
arXiv poster
Algorithms:
An optimal algorithm for strongly convex minimization under affine constraints
24th International Conference on Artificial Intelligence and Statistics (AISTATS 2022)
arXiv poster
Algorithms:
accelerated PAPC
[158] Zhen Shi, Nicolas Loizou, Peter Richtárik and Martin Takáč
AI-SARAH: Adaptive and implicit stochastic recursive gradient methods
Transactions on Machine Learning Research (TMLR 2023)
arXiv
Algorithms:
AI-SARAH: Adaptive and implicit stochastic recursive gradient methods
Transactions on Machine Learning Research (TMLR 2023)
arXiv
Algorithms:
AI-SARAH
[157] Dmitry Kovalev, Egor Shulgin, Peter Richtárik, Alexander Rogozin and Alexander Gasnikov
ADOM: Accelerated decentralized optimization method for time-varying networks
38th International Conference on Machine Learning (ICML 2021)
NSF-TRIPODS Workshop: Communication Efficient Distributed Optimization
arXiv video poster
Algorithms:
ADOM: Accelerated decentralized optimization method for time-varying networks
38th International Conference on Machine Learning (ICML 2021)
NSF-TRIPODS Workshop: Communication Efficient Distributed Optimization
arXiv video poster
Algorithms:
ADOM
[156] Konstantin Mishchenko, Bokun Wang, Dmitry Kovalev and Peter Richtárik
IntSGD: Floatless compression of stochastic gradients
10th International Conference on Learning Representations (ICLR 2022)
ICLR 2022 Spotlight paper
arXiv video poster
Algorithms:
IntSGD: Floatless compression of stochastic gradients
10th International Conference on Learning Representations (ICLR 2022)
ICLR 2022 Spotlight paper
arXiv video poster
Algorithms:
IntSGD, IntDIANA
[155] Eduard Gorbunov, Konstantin Burlachenko, Zhize Li and Peter Richtárik
MARINA: faster non-convex distributed learning with compression
38th International Conference on Machine Learning (ICML 2021)
NSF-TRIPODS Workshop: Communication Efficient Distributed Optimization
arXiv video 1 video 2 poster
Algorithms:
MARINA: faster non-convex distributed learning with compression
38th International Conference on Machine Learning (ICML 2021)
NSF-TRIPODS Workshop: Communication Efficient Distributed Optimization
arXiv video 1 video 2 poster
Algorithms:
MARINA, VR-MARINA, PP-MARINA
[154] Mher Safaryan, Filip Hanzely and Peter Richtárik
Smoothness matrices beat smoothness constants: better communication compression techniques for distributed optimization
Advances in Neural Information Processing Systems 34 (NeurIPS 2021)
ICLR Workshop: Distributed and Private Machine Learning
NSF-TRIPODS Workshop: Communication Efficient Distributed Optimization
arXiv video poster
Algorithms:
Smoothness matrices beat smoothness constants: better communication compression techniques for distributed optimization
Advances in Neural Information Processing Systems 34 (NeurIPS 2021)
ICLR Workshop: Distributed and Private Machine Learning
NSF-TRIPODS Workshop: Communication Efficient Distributed Optimization
arXiv video poster
Algorithms:
DCGD+, DIANA+, ADIANA+
[153] Rustem Islamov, Xun Qian and Peter Richtárik
Distributed second order methods with fast rates and compressed communication
38th International Conference on Machine Learning (ICML 2021)
NSF-TRIPODS Workshop: Communication Efficient Distributed Optimization
arXiv slides video 1 video 2 video 3 poster
Algorithms:
Distributed second order methods with fast rates and compressed communication
38th International Conference on Machine Learning (ICML 2021)
NSF-TRIPODS Workshop: Communication Efficient Distributed Optimization
arXiv slides video 1 video 2 video 3 poster
Algorithms:
NS, MN, NL1, NL2, CNL
[152] Konstantin Mishchenko, Ahmed Khaled and Peter Richtárik
Proximal and federated random reshuffling
39th International Conference on Machine Learning (ICML 2022)
NSF-TRIPODS Workshop: Communication Efficient Distributed Optimization
arXiv video
Algorithms:
Proximal and federated random reshuffling
39th International Conference on Machine Learning (ICML 2022)
NSF-TRIPODS Workshop: Communication Efficient Distributed Optimization
arXiv video
Algorithms:
ProxRR, FedRR
Prepared in 2020
[151] Samuel Horváth, Aaron Klein, Peter Richtárik and Cedric Archambeau
Hyperparameter transfer learning with adaptive complexity
The 24th International Conference on Artificial Intelligence and Statistics (AISTATS 2021)
arXiv poster
Algorithms:
Hyperparameter transfer learning with adaptive complexity
The 24th International Conference on Artificial Intelligence and Statistics (AISTATS 2021)
arXiv poster
Algorithms:
ABRAC
[150] Xun Qian, Hanze Dong, Peter Richtárik and Tong Zhang
Error compensated loopless SVRG for distributed optimization
OPT2020: 12th Annual Workshop on Optimization for Machine Learning (NeurIPS 2020 Workshop)
arXiv poster
Algorithms:
Error compensated loopless SVRG for distributed optimization
OPT2020: 12th Annual Workshop on Optimization for Machine Learning (NeurIPS 2020 Workshop)
arXiv poster
Algorithms:
EC-LSVRG
[149] Xun Qian, Hanze Dong, Peter Richtárik and Tong Zhang
Error compensated proximal SGD and RDA
OPT2020: 12th Annual Workshop on Optimization for Machine Learning (NeurIPS 2020 Workshop)
poster
Algorithms:
Error compensated proximal SGD and RDA
OPT2020: 12th Annual Workshop on Optimization for Machine Learning (NeurIPS 2020 Workshop)
poster
Algorithms:
EC-SGD, EC-RDA
[148] Eduard Gorbunov, Filip Hanzely, and Peter Richtárik
Local SGD: unified theory and new efficient methods
The 24th International Conference on Artificial Intelligence and Statistics (AISTATS 2021)
arXiv video poster
Algorithms:
Local SGD: unified theory and new efficient methods
The 24th International Conference on Artificial Intelligence and Statistics (AISTATS 2021)
arXiv video poster
Algorithms:
S-Local-SVRG
[147] Dmitry Kovalev, Anastasia Koloskova, Martin Jaggi, Peter Richtárik, and Sebastian U. Stich
A linearly convergent algorithm for decentralized optimization: sending less bits for free!
The 24th International Conference on Artificial Intelligence and Statistics (AISTATS 2021)
arXiv video poster
A linearly convergent algorithm for decentralized optimization: sending less bits for free!
The 24th International Conference on Artificial Intelligence and Statistics (AISTATS 2021)
arXiv video poster
[146] Wenlin Chen, Samuel Horváth, and Peter Richtárik
Optimal client sampling for federated learning
Transactions on Machine Learning Research (TMLR 2022)
Privacy Preserving Machine Learning (NeurIPS 2020 Workshop)
arXiv
Algorithms:
Optimal client sampling for federated learning
Transactions on Machine Learning Research (TMLR 2022)
Privacy Preserving Machine Learning (NeurIPS 2020 Workshop)
arXiv
Algorithms:
OCS, AOCS
[145] Eduard Gorbunov, Dmitry Kovalev, Dmitry Makarenko, and Peter Richtárik
Linearly converging error compensated SGD
Advances in Neural Information Processing Systems 33 (NeurIPS 2020)
arXiv video poster
Algorithms:
Linearly converging error compensated SGD
Advances in Neural Information Processing Systems 33 (NeurIPS 2020)
arXiv video poster
Algorithms:
EC-SGD-DIANA, EC-LSVRG-DIANA, EC-LSVRGstar, ...
[144] Alyazeed Albasyoni, Mher Safaryan, Laurent Condat, and Peter Richtárik
Optimal gradient compression for distributed and federated learning
SpicyFL 2020: NeurIPS Workshop on Scalability, Privacy, and Security in Federated Learning
arXiv video poster
Optimal gradient compression for distributed and federated learning
SpicyFL 2020: NeurIPS Workshop on Scalability, Privacy, and Security in Federated Learning
arXiv video poster
[143] Filip Hanzely, Slavomír Hanzely, Samuel Horváth, and Peter Richtárik
Lower bounds and optimal algorithms for personalized federated learning
Advances in Neural Information Processing Systems 33 (NeurIPS 2020)
arXiv video
Algorithms:
Lower bounds and optimal algorithms for personalized federated learning
Advances in Neural Information Processing Systems 33 (NeurIPS 2020)
arXiv video
Algorithms:
APGD1, APGD2, IAPGD, AL2SGD+
[142] Laurent Condat, Grigory Malinovsky, and Peter Richtárik
Distributed proximal splitting algorithms with rates and acceleration
Frontiers in Signal Processing, section Signal Processing for Communications, 2022
OPT2020: 12th Annual Workshop on Optimization for Machine Learning (NeurIPS 2020 Workshop)
Spotlight Talk
arXiv poster
Algorithms:
Distributed proximal splitting algorithms with rates and acceleration
Frontiers in Signal Processing, section Signal Processing for Communications, 2022
OPT2020: 12th Annual Workshop on Optimization for Machine Learning (NeurIPS 2020 Workshop)
Spotlight Talk
arXiv poster
Algorithms:
PD3O, PDDY, distributed PD3O, distributed PDDY
[141] Robert M. Gower, Mark Schmidt, Francis Bach and Peter Richtárik
Variance-reduced methods for machine learning
Proceedings of the IEEE 108 (11):1968--1983, 2020
arXiv
Algorithms:
Variance-reduced methods for machine learning
Proceedings of the IEEE 108 (11):1968--1983, 2020
arXiv
Algorithms:
SAG, SAGA, SVRG, SDCA
[140] Xun Qian, Peter Richtárik, and Tong Zhang
Error compensated distributed SGD can be accelerated
Advances in Neural Information Processing Systems 34 (NeurIPS 2021)
OPT2020: 12th Annual Workshop on Optimization for Machine Learning (NeurIPS 2020 Workshop)
arXiv poster
Algorithms:
Error compensated distributed SGD can be accelerated
Advances in Neural Information Processing Systems 34 (NeurIPS 2021)
OPT2020: 12th Annual Workshop on Optimization for Machine Learning (NeurIPS 2020 Workshop)
arXiv poster
Algorithms:
ECLK
[139] Albert S. Berahas, Majid Jahani, Peter Richtárik, and Martin Takáč
Quasi-Newton methods for deep learning: forget the past, just sample
Optimization Methods and Software 37(5):1668-1704, 2022
2022 Charles Broyden Prize
arXiv
Algorithms:
Quasi-Newton methods for deep learning: forget the past, just sample
Optimization Methods and Software 37(5):1668-1704, 2022
2022 Charles Broyden Prize
arXiv
Algorithms:
S-LBFGS, S-LSR1
[138] Zhize Li, Hongyan Bao, Xiangliang Zhang and Peter Richtárik
PAGE: A simple and optimal probabilistic gradient estimator for nonconvex optimization
38th International Conference on Machine Learning (ICML 2021)
OPT2020: 12th Annual Workshop on Optimization for Machine Learning (NeurIPS 2020 Workshop) (Spotlight Talk)
arXiv video poster
Algorithms:
PAGE: A simple and optimal probabilistic gradient estimator for nonconvex optimization
38th International Conference on Machine Learning (ICML 2021)
OPT2020: 12th Annual Workshop on Optimization for Machine Learning (NeurIPS 2020 Workshop) (Spotlight Talk)
arXiv video poster
Algorithms:
PAGE
[137] Dmitry Kovalev, Adil Salim, and Peter Richtárik
Optimal and practical algorithms for smooth and strongly convex decentralized optimization
Advances in Neural Information Processing Systems 33 (NeurIPS 2020)
arXiv
Algorithms:
Optimal and practical algorithms for smooth and strongly convex decentralized optimization
Advances in Neural Information Processing Systems 33 (NeurIPS 2020)
arXiv
Algorithms:
APAPC, OPAPC, Algorithm 3
[136] Ahmed Khaled, Othmane Sebbouh, Nicolas Loizou, Robert M. Gower, and Peter Richtárik
Unified analysis of stochastic gradient methods for composite convex and smooth optimization
Journal of Optimization Theory and Applications 199:499-540, 2023
arXiv
Algorithms:
Unified analysis of stochastic gradient methods for composite convex and smooth optimization
Journal of Optimization Theory and Applications 199:499-540, 2023
arXiv
Algorithms:
SGD
[135] Samuel Horváth and Peter Richtárik
A better alternative to error feedback for communication-efficient distributed learning
9th International Conference on Learning Representations (ICLR 2021)
SpicyFL 2020: NeurIPS Workshop on Scalability, Privacy, and Security in Federated Learning
The Best Paper Award at NeurIPS-20 Workshop on Scalability, Privacy, and Security in Federated Learning
arXiv poster
Algorithms:
A better alternative to error feedback for communication-efficient distributed learning
9th International Conference on Learning Representations (ICLR 2021)
SpicyFL 2020: NeurIPS Workshop on Scalability, Privacy, and Security in Federated Learning
The Best Paper Award at NeurIPS-20 Workshop on Scalability, Privacy, and Security in Federated Learning
arXiv poster
Algorithms:
DCSGD
[134] Adil Salim and Peter Richtárik
Primal dual interpretation of the proximal stochastic gradient Langevin algorithm
Advances in Neural Information Processing Systems 33 (NeurIPS 2020)
arXiv
Algorithms:
Primal dual interpretation of the proximal stochastic gradient Langevin algorithm
Advances in Neural Information Processing Systems 33 (NeurIPS 2020)
arXiv
Algorithms:
PGSLA
[133] Zhize Li and Peter Richtárik
A unified analysis of stochastic gradient methods for nonconvex federated optimization
SpicyFL 2020: NeurIPS Workshop on Scalability, Privacy, and Security in Federated Learning
arXiv video
Algorithms:
A unified analysis of stochastic gradient methods for nonconvex federated optimization
SpicyFL 2020: NeurIPS Workshop on Scalability, Privacy, and Security in Federated Learning
arXiv video
Algorithms:
DC-GD, DC-SGD, DC-LSVRG, DC-SAGA, DIANA-GD, DIANA-SGD, DIANA-LSVRG, DIANA-SAGA
[132] Konstantin Mishchenko, Ahmed Khaled, and Peter Richtárik
Random reshuffling: simple analysis with vast improvements
Advances in Neural Information Processing Systems 33 (NeurIPS 2020)
arXiv video poster code
Algorithms:
Random reshuffling: simple analysis with vast improvements
Advances in Neural Information Processing Systems 33 (NeurIPS 2020)
arXiv video poster code
Algorithms:
RR, SO, IG
[131] Motasem Alfarra, Slavomír Hanzely, Alyazeed Albasyoni, Bernard Ghanem, and Peter Richtárik
Adaptive learning of the optimal mini-batch size of SGD
OPT2020: 12th Annual Workshop on Optimization for Machine Learning (NeurIPS 2020 Workshop)
arXiv poster
Algorithms:
Adaptive learning of the optimal mini-batch size of SGD
OPT2020: 12th Annual Workshop on Optimization for Machine Learning (NeurIPS 2020 Workshop)
arXiv poster
Algorithms:
SGD with Adaptive Batch size
[130] Adil Salim, Laurent Condat, Konstantin Mishchenko, and Peter Richtárik
Dualize, split, randomize: fast nonsmooth optimization algorithms
Journal of Optimization Theory and Applications 195: 102-130, 2022
OPT2020: 12th Annual Workshop on Optimization for Machine Learning (NeurIPS 2020 Workshop)
arXiv poster
Algorithms:
Dualize, split, randomize: fast nonsmooth optimization algorithms
Journal of Optimization Theory and Applications 195: 102-130, 2022
OPT2020: 12th Annual Workshop on Optimization for Machine Learning (NeurIPS 2020 Workshop)
arXiv poster
Algorithms:
PDDY, SPDDY, SPD3O, SPAPC
[129] Atal Narayan Sahu, Aritra Dutta, Aashutosh Tiwari, and Peter Richtárik
On the convergence analysis of asynchronous SGD for solving consistent linear systems
Linear Algebra and its Applications, 2022
arXiv
Algorithms:
On the convergence analysis of asynchronous SGD for solving consistent linear systems
Linear Algebra and its Applications, 2022
arXiv
Algorithms:
DASGD
[128] Grigory Malinovsky, Dmitry Kovalev, Elnur Gasanov, Laurent Condat, and Peter Richtárik
From local SGD to local fixed point methods for federated learning
37th International Conference on Machine Learning (ICML 2020)
arXiv video
Algorithms:
From local SGD to local fixed point methods for federated learning
37th International Conference on Machine Learning (ICML 2020)
arXiv video
Algorithms:
LDFPM, RDFPM
[127] Aleksandr Beznosikov, Samuel Horváth, Peter Richtárik and Mher Safaryan
On biased compression for distributed learning
Accepted to Journal of Machine Learning Research, 2022
SpicyFL 2020: NeurIPS Workshop on Scalability, Privacy, and Security in Federated Learning
arXiv poster
Algorithms:
On biased compression for distributed learning
Accepted to Journal of Machine Learning Research, 2022
SpicyFL 2020: NeurIPS Workshop on Scalability, Privacy, and Security in Federated Learning
arXiv poster
Algorithms:
CGD, Distributed SGD with Error Feedback
[126] Zhize Li, Dmitry Kovalev, Xun Qian and Peter Richtárik
Acceleration for compressed gradient descent in distributed and federated optimization
37th International Conference on Machine Learning (ICML 2020)
arXiv
Algorithms:
Acceleration for compressed gradient descent in distributed and federated optimization
37th International Conference on Machine Learning (ICML 2020)
arXiv
Algorithms:
ACGD, ADIANA
[125] Dmitry Kovalev, Robert M. Gower, Peter Richtárik and Alexander Rogozin
Fast linear convergence of randomized BFGS
arXiv
Algorithms:
Fast linear convergence of randomized BFGS
arXiv
Algorithms:
RBFGS
[124] Filip Hanzely, Nikita Doikov, Peter Richtárik and Yurii Nesterov
Stochastic subspace cubic Newton method
37th International Conference on Machine Learning (ICML 2020)
arXiv
Algorithms:
Stochastic subspace cubic Newton method
37th International Conference on Machine Learning (ICML 2020)
arXiv
Algorithms:
SSCN
[123] Mher Safaryan, Egor Shulgin and Peter Richtárik
Uncertainty principle for communication compression in distributed and federated learning and the search for an optimal compressor
Information and Inference: A Journal of the IMA, 1--24, 2021
arXiv
Algorithms:
Uncertainty principle for communication compression in distributed and federated learning and the search for an optimal compressor
Information and Inference: A Journal of the IMA, 1--24, 2021
arXiv
Algorithms:
KC
[122] Filip Hanzely and Peter Richtárik
Federated learning of a mixture of global and local models
SpicyFL 2020: NeurIPS Workshop on Scalability, Privacy, and Security in Federated Learning
arXiv slides video poster
Algorithms:
Federated learning of a mixture of global and local models
SpicyFL 2020: NeurIPS Workshop on Scalability, Privacy, and Security in Federated Learning
arXiv slides video poster
Algorithms:
L2GD, L2SGD+
[121] Samuel Horváth, Lihua Lei, Peter Richtárik and Michael I. Jordan
Adaptivity of stochastic gradient methods for nonconvex optimization
OPT2020: 12th Annual Workshop on Optimization for Machine Learning (NeurIPS 2020 Workshop)
SIAM Journal on Mathematics of Data Science 4(2):634--648, 2022
arXiv poster
Algorithms:
Adaptivity of stochastic gradient methods for nonconvex optimization
OPT2020: 12th Annual Workshop on Optimization for Machine Learning (NeurIPS 2020 Workshop)
SIAM Journal on Mathematics of Data Science 4(2):634--648, 2022
arXiv poster
Algorithms:
Geometrized SARAH
[120] Filip Hanzely, Dmitry Kovalev and Peter Richtárik
Variance reduced coordinate descent with acceleration: new method with a surprising application to finite-sum problems
37th International Conference on Machine Learning (ICML 2020)
arXiv
Algorithms:
Variance reduced coordinate descent with acceleration: new method with a surprising application to finite-sum problems
37th International Conference on Machine Learning (ICML 2020)
arXiv
Algorithms:
ASVRCD
[119] Ahmed Khaled and Peter Richtárik
Better theory for SGD in the nonconvex world
Transactions on Machine Learning Research (TMLR 2022)
arXiv
Algorithms:
Better theory for SGD in the nonconvex world
Transactions on Machine Learning Research (TMLR 2022)
arXiv
Algorithms:
SGDPrepared in 2019
[118] Ahmed Khaled, Konstantin Mishchenko and Peter Richtárik
Tighter theory for local SGD on identical and heterogeneous data
The 23rd International Conference on Artificial Intelligence and Statistics (AISTATS 2020)
arXiv
Algorithms:
Tighter theory for local SGD on identical and heterogeneous data
The 23rd International Conference on Artificial Intelligence and Statistics (AISTATS 2020)
arXiv
Algorithms:
Local SGD
[117] Sélim Chraibi, Ahmed Khaled, Dmitry Kovalev, Adil Salim,
Peter Richtárik and Martin Takáč
Distributed fixed point methods with compressed iterates
arXiv preprint
Algorithms:
Distributed fixed point methods with compressed iterates
arXiv preprint
Algorithms:
FPMCI, VR-FPMCI
[116] Samuel Horváth, Chen-Yu Ho, Ľudovít Horváth, Atal Narayan Sahu, Marco Canini and Peter Richtárik
IntML: Natural compression for distributed deep learning
Workshop on AI Systems at Symposium on Operating Systems Principles 2019 (SOSP'19)
arXiv pdf
Algorithms:
IntML: Natural compression for distributed deep learning
Workshop on AI Systems at Symposium on Operating Systems Principles 2019 (SOSP'19)
arXiv pdf
Algorithms:
NC, natural dithering
[115] Dmitry Kovalev, Konstantin Mishchenko and Peter Richtárik
Stochastic Newton and cubic Newton methods with simple local linear-quadratic rates
NeurIPS 2019 Workshop Beyond First Order Methods in ML
arXiv poster
Algorithms:
Stochastic Newton and cubic Newton methods with simple local linear-quadratic rates
NeurIPS 2019 Workshop Beyond First Order Methods in ML
arXiv poster
Algorithms:
SN, SCN
[114] Ahmed Khaled, Konstantin Mishchenko and Peter Richtárik
Better communication complexity for local SGD
NeurIPS 2019 Workshop on Federated Learning for Data Privacy and Confidentiality
arXiv poster
Algorithms:
Better communication complexity for local SGD
NeurIPS 2019 Workshop on Federated Learning for Data Privacy and Confidentiality
arXiv poster
Algorithms:
local SGD
[113] Ahmed Khaled and Peter Richtárik
Gradient descent with compressed iterates
NeurIPS 2019 Workshop on Federated Learning for Data Privacy and Confidentiality
arXiv poster
Algorithms:
Gradient descent with compressed iterates
NeurIPS 2019 Workshop on Federated Learning for Data Privacy and Confidentiality
arXiv poster
Algorithms:
GDCI
[112] Ahmed Khaled, Konstantin Mishchenko and Peter Richtárik
First analysis of local GD on heterogeneous data
NeurIPS 2019 Workshop on Federated Learning for Data Privacy and Confidentiality
arXiv
Algorithms:
First analysis of local GD on heterogeneous data
NeurIPS 2019 Workshop on Federated Learning for Data Privacy and Confidentiality
arXiv
Algorithms:
local GD
[111] Jinhui Xiong, Peter Richtárik and Wolfgang Heidrich
Stochastic convolutional sparse coding
International Symposium on Vision, Modeling and Visualization 2019
VMV Best Paper Award, 2019 link
arXiv
Algorithms:
Stochastic convolutional sparse coding
International Symposium on Vision, Modeling and Visualization 2019
VMV Best Paper Award, 2019 link
arXiv
Algorithms:
SBCSC, SOCSC
[110] Xun Qian, Zheng Qu and Peter Richtárik
L-SVRG and L-Katyusha with arbitrary sampling
Journal of Machine Learning Research 22(112):1−47, 2021
arXiv video
Algorithms:
L-SVRG and L-Katyusha with arbitrary sampling
Journal of Machine Learning Research 22(112):1−47, 2021
arXiv video
Algorithms:
L-SVRG, L-Katyusha
[109] Xun Qian, Alibek Sailanbayev, Konstantin Mishchenko and
Peter Richtárik
MISO is making a comeback with better proofs and rates
arXiv
Algorithms:
MISO is making a comeback with better proofs and rates
arXiv
Algorithms:
MISO
[108] Eduard Gorbunov, Adel Bibi, Ozan Sezer, El Houcine Bergou and Peter Richtárik
A stochastic derivative free optimization method with momentum
8th International Conference on Learning Representations (ICLR 2020)
arXiv poster
Algorithms:
A stochastic derivative free optimization method with momentum
8th International Conference on Learning Representations (ICLR 2020)
arXiv poster
Algorithms:
SMTP
[107] Mher Safaryan and Peter Richtárik
Stochastic Sign Descent Methods: New Algorithms and Better Theory
38th International Conference on Machine Learning (ICML 2021)
OPT2020: 12th Annual Workshop on Optimization for Machine Learning (NeurIPS 2020 Workshop)
arXiv poster
Algorithms:
Stochastic Sign Descent Methods: New Algorithms and Better Theory
38th International Conference on Machine Learning (ICML 2021)
OPT2020: 12th Annual Workshop on Optimization for Machine Learning (NeurIPS 2020 Workshop)
arXiv poster
Algorithms:
signSGD, signSGDmaj
[106] Adil Salim, Dmitry Kovalev and Peter Richtárik
Stochastic proximal Langevin algorithm: potential splitting and nonasymptotic rates
33rd Conference on Neural Information Processing Systems (NeurIPS 2019)
arXiv poster
Algorithms:
Stochastic proximal Langevin algorithm: potential splitting and nonasymptotic rates
33rd Conference on Neural Information Processing Systems (NeurIPS 2019)
arXiv poster
Algorithms:
SPLA
[105] Aritra Dutta, El Houcine Bergou, Yunming Xiao, Marco Canini and Peter Richtárik
Direct nonlinear acceleration
EURO Journal on Computational Optimization 10, 2022, 100047
arXiv
Algorithms:
Direct nonlinear acceleration
EURO Journal on Computational Optimization 10, 2022, 100047
arXiv
Algorithms:
DNA
[104] Konstantin Mishchenko and Peter Richtárik
A stochastic decoupling method for minimizing the sum of smooth and non-smooth functions
arXiv
Algorithms:
A stochastic decoupling method for minimizing the sum of smooth and non-smooth functions
arXiv
Algorithms:
SDM
[103] Konstantin Mishchenko, Dmitry Kovalev, Egor Shulgin, Peter Richtárik and Yura Malitsky
Revisiting stochastic extragradient
The 23rd International Conference on Artificial Intelligence and Statistics (AISTATS 2020)
NeuriPS 2019 Workshop on Smooth Games Optimization and Machine Learning
arXiv
Algorithms:
Revisiting stochastic extragradient
The 23rd International Conference on Artificial Intelligence and Statistics (AISTATS 2020)
NeuriPS 2019 Workshop on Smooth Games Optimization and Machine Learning
arXiv
Algorithms:
stochastic extragradient
[102] Filip Hanzely and Peter Richtárik
One method to rule them all: variance reduction for data, parameters and many new methods
arXiv poster
Algorithms:
One method to rule them all: variance reduction for data, parameters and many new methods
arXiv poster
Algorithms:
GJS + 17 algorithms
[101] Eduard Gorbunov, Filip Hanzely and Peter Richtárik
A unified theory of SGD: variance reduction, sampling, quantization and coordinate descent
The 23rd International Conference on Artificial Intelligence and Statistics (AISTATS 2020)
arXiv
Algorithms:
A unified theory of SGD: variance reduction, sampling, quantization and coordinate descent
The 23rd International Conference on Artificial Intelligence and Statistics (AISTATS 2020)
arXiv
Algorithms:
SGD-MB, SGD-star, N-SAGA, N-SEGA, Q-SGD-SR
[100] Samuel Horváth, Chen-Yu Ho, Ľudovít Horváth, Atal
Narayan Sahu, Marco Canini and Peter Richtárik
Natural compression for distributed deep learning
Mathematical and Scientific Machine Learning 2022 (MSML 2022)
arXiv poster
Algorithms:
Natural compression for distributed deep learning
Mathematical and Scientific Machine Learning 2022 (MSML 2022)
arXiv poster
Algorithms:
NC, natural dithering
[99] Robert M. Gower, Dmitry Kovalev, Felix Lieder and Peter
Richtárik
RSN: Randomized Subspace Newton
33rd Conference on Neural Information Processing Systems (NeurIPS 2019)
arXiv poster
Algorithms:
RSN: Randomized Subspace Newton
33rd Conference on Neural Information Processing Systems (NeurIPS 2019)
arXiv poster
Algorithms:
RSN
[98] Aritra Dutta, Filip Hanzely, Jingwei Liang and Peter
Richtárik
Best pair formulation & accelerated scheme for non-convex principal component pursuit
IEEE Transactions on Signal Processing 68:6128-6141, 2020
arXiv
Algorithms:
Best pair formulation & accelerated scheme for non-convex principal component pursuit
IEEE Transactions on Signal Processing 68:6128-6141, 2020
arXiv
Algorithms:
accelerated proximal gradient
[97] Nicolas Loizou and Peter Richtárik
Revisiting randomized gossip algorithms: general framework, convergence rates and novel block and accelerated protocols
IEEE Transactions on Information Theory 67(12):8300--8324, 2021
arXiv
Algorithms:
Revisiting randomized gossip algorithms: general framework, convergence rates and novel block and accelerated protocols
IEEE Transactions on Information Theory 67(12):8300--8324, 2021
arXiv
Algorithms:
block gossip, accelerated gossip, dual gossip
[96] Nicolas Loizou and Peter Richtárik
Convergence analysis of inexact randomized iterative methods
SIAM Journal on Scientific Computing 42(6), A3979–A4016, 2020
arXiv
Algorithms:
Convergence analysis of inexact randomized iterative methods
SIAM Journal on Scientific Computing 42(6), A3979–A4016, 2020
arXiv
Algorithms:
iBasic, iSDSA, iSGD, iSPM, iRBK, iRBCD
[95] Amedeo Sapio, Marco Canini, Chen-Yu Ho, Jacob Nelson,
Panos Kalnis, Changhoon Kim, Arvind Krishnamurthy, Masoud
Moshref, Dan R. K. Ports and Peter Richtárik
Scaling distributed machine learning with in-network aggregation
The 18th USENIX Symposium on Networked Systems Design and Implementation (NSDI '21 Fall)
arXiv
Algorithms:
Scaling distributed machine learning with in-network aggregation
The 18th USENIX Symposium on Networked Systems Design and Implementation (NSDI '21 Fall)
arXiv
Algorithms:
SwitchML
[94] Samuel Horváth, Dmitry Kovalev, Konstantin Mishchenko, Peter Richtárik and Sebastian Stich
Stochastic distributed learning with gradient quantization and double variance reduction
Optimization Methods and Software 38(1):91-106, 2023
2023 Charles Broyden Prize
arXiv
Algorithms:
Stochastic distributed learning with gradient quantization and double variance reduction
Optimization Methods and Software 38(1):91-106, 2023
2023 Charles Broyden Prize
arXiv
Algorithms:
DIANA, VR-DIANA, SVRG-DIANA
[93] El Houcine Bergou, Eduard Gorbunov and Peter Richtárik
Stochastic three points method for unconstrained smooth minimization
SIAM Journal on Optimization 30(4):2726-2749, 2020
arXiv
Algorithms:
Stochastic three points method for unconstrained smooth minimization
SIAM Journal on Optimization 30(4):2726-2749, 2020
arXiv
Algorithms:
STP
[92] Adel Bibi, El Houcine Bergou, Ozan Sener, Bernard Ghanem and Peter Richtárik
A stochastic derivative-free optimization method with importance sampling
34th AAAI Conference on Artificial Intelligence (AAAI 2020)
arXiv poster
Algorithms:
A stochastic derivative-free optimization method with importance sampling
34th AAAI Conference on Artificial Intelligence (AAAI 2020)
arXiv poster
Algorithms:
STP_IS
[91] Konstantin Mishchenko, Filip Hanzely and Peter Richtárik
99% of distributed optimization is a waste of time: the issue and how to fix it
36th Conference on Uncertainty in Artificial Intelligence (UAI 2020)
arXiv
Algorithms:
99% of distributed optimization is a waste of time: the issue and how to fix it
36th Conference on Uncertainty in Artificial Intelligence (UAI 2020)
arXiv
Algorithms:
IBCD, ISAGA, ISGD, IASGD, ISEGA
[90] Konstantin Mishchenko, Eduard Gorbunov, Martin Takáč and Peter Richtárik
Distributed learning with compressed gradient differences
Optimization Methods and Software 40(5):1181--1196, 2025
arXiv
Algorithms:
Distributed learning with compressed gradient differences
Optimization Methods and Software 40(5):1181--1196, 2025
arXiv
Algorithms:
DIANA
[89] Robert Mansel Gower, Nicolas Loizou, Xun Qian, Alibek Sailanbayev, Egor Shulgin and Peter Richtárik
SGD: general analysis and improved rates
Proceedings of the 36th International Conference on Machine Learning, PMLR 97:5200-5209, 2019
arXiv video poster
Algorithms:
SGD: general analysis and improved rates
Proceedings of the 36th International Conference on Machine Learning, PMLR 97:5200-5209, 2019
arXiv video poster
Algorithms:
SGD-AS
[88] Dmitry Kovalev, Samuel Horváth and Peter Richtárik
Don’t jump through hoops and remove those loops: SVRG and Katyusha are better without the outer loop
31st International Conference on Learning Theory (ALT 2020)
arXiv
Algorithms:
Don’t jump through hoops and remove those loops: SVRG and Katyusha are better without the outer loop
31st International Conference on Learning Theory (ALT 2020)
arXiv
Algorithms:
L-SVRG, L-Katyusha
[87] Xun Qian, Zheng Qu and Peter Richtárik
SAGA with arbitrary sampling
Proceedings of the 36th International Conference on Machine Learning, PMLR 97:5190-5199, 2019
arXiv poster
Algorithms:
SAGA with arbitrary sampling
Proceedings of the 36th International Conference on Machine Learning, PMLR 97:5190-5199, 2019
arXiv poster
Algorithms:
SAGA-AS
Prepared in 2018
[86] Lam M. Nguyen, Phuong Ha Nguyen, P. Richtárik, Katya Scheinberg, Martin Takáč and Marten van Dijk
New convergence aspects of stochastic gradient algorithms
Journal of Machine Learning Research 20(176):1-49, 2019
arXiv
Algorithms:
New convergence aspects of stochastic gradient algorithms
Journal of Machine Learning Research 20(176):1-49, 2019
arXiv
Algorithms:
SGD, Hogwild!
[85] Filip Hanzely, Jakub Konečný, Nicolas Loizou, Peter Richtárik and Dmitry Grishchenko
A privacy preserving randomized gossip algorithm via controlled noise insertion
NeurIPS Privacy Preserving Machine Learning Workshop, 2018
arXiv poster
Algorithms:
A privacy preserving randomized gossip algorithm via controlled noise insertion
NeurIPS Privacy Preserving Machine Learning Workshop, 2018
arXiv poster
Algorithms:
Private Gossip with Controlled Noise Insertion
[84] Konstantin Mishchenko and Peter Richtárik
A stochastic penalty model for convex and nonconvex optimization with big constraints
arXiv poster
Algorithms:
A stochastic penalty model for convex and nonconvex optimization with big constraints
arXiv poster
Algorithms:
SGD, Increasing Penalty Method
[83] Nicolas Loizou, Michael G. Rabbat and Peter Richtárik
Provably accelerated randomized gossip algorithms
2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP 2019)
arXiv
Algorithms:
Provably accelerated randomized gossip algorithms
2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP 2019)
arXiv
Algorithms:
AccGossip
[82] Filip Hanzely and Peter Richtárik
Accelerated coordinate descent with arbitrary sampling and best rates for minibatches
22nd International Conference on Artificial Intelligence and Statistics (2019
arXiv poster
Algorithms:
Accelerated coordinate descent with arbitrary sampling and best rates for minibatches
22nd International Conference on Artificial Intelligence and Statistics (2019
arXiv poster
Algorithms:
ACD
[81] Samuel Horváth and Peter Richtárik
Nonconvex variance reduced optimization with arbitrary sampling
Proceedings of the 36th International Conference on Machine Learning, PMLR 97:2781-2789, 2019
Horváth: Best DS3 Poster Award, Paris, 2018 (link)
arXiv poster
Algorithms:
Nonconvex variance reduced optimization with arbitrary sampling
Proceedings of the 36th International Conference on Machine Learning, PMLR 97:2781-2789, 2019
Horváth: Best DS3 Poster Award, Paris, 2018 (link)
arXiv poster
Algorithms:
SVRG, SAGA, SARAH
[80] Filip Hanzely, Konstantin Mishchenko and Peter Richtárik
SEGA: Variance reduction via gradient sketching
Advances in Neural Information Processing Systems 31:2082-2093, 2018
arXiv slides video poster
Algorithms:
SEGA: Variance reduction via gradient sketching
Advances in Neural Information Processing Systems 31:2082-2093, 2018
arXiv slides video poster
Algorithms:
SEGA
[79] Filip Hanzely, Peter Richtárik and Lin Xiao
Accelerated Bregman proximal gradient methods for relatively smooth convex optimization
Computational Optimization and Applications 79:405–440, 2021
arXiv
Algorithms:
Accelerated Bregman proximal gradient methods for relatively smooth convex optimization
Computational Optimization and Applications 79:405–440, 2021
arXiv
Algorithms:
ABPG, ABDA
[78] Jakub Mareček, Peter Richtárik and Martin Takáč
Matrix completion under interval uncertainty: highlights
Lecture Notes in Computer Science, ECML-PKDD 2018
pdf
Algorithms:
Matrix completion under interval uncertainty: highlights
Lecture Notes in Computer Science, ECML-PKDD 2018
Algorithms:
MACO
[77] Nicolas Loizou and Peter Richtárik
Accelerated gossip via stochastic heavy ball method
56th Annual Allerton Conference on Communication, Control, and Computing, 927-934, 2018
Press coverage [KAUST Discovery]
arXiv poster
Algorithms:
Accelerated gossip via stochastic heavy ball method
56th Annual Allerton Conference on Communication, Control, and Computing, 927-934, 2018
Press coverage [KAUST Discovery]
arXiv poster
Algorithms:
SHB, mRK, mRBK
[76] Adel Bibi, Alibek Sailanbayev, Bernard Ghanem, Robert Mansel Gower and Peter Richtárik
Improving SAGA via a probabilistic interpolation with gradient descent
arXiv
Algorithms:
Improving SAGA via a probabilistic interpolation with gradient descent
arXiv
Algorithms:
SAGD
[75] Aritra Dutta, Filip Hanzely and Peter Richtárik
A nonconvex projection method for robust PCA
33rd AAAI Conference on Artificial Intelligence (AAAI 2019)
arXiv
Algorithms:
A nonconvex projection method for robust PCA
33rd AAAI Conference on Artificial Intelligence (AAAI 2019)
arXiv
Algorithms:
alternating projection for RPCA, alternating projection for RMC
[74] Robert M. Gower, Peter Richtárik and Francis Bach
Stochastic quasi-gradient methods: variance reduction via Jacobian sketching
Mathematical Programming 188:135–192, 2021
arXiv slides video
Algorithms:
Stochastic quasi-gradient methods: variance reduction via Jacobian sketching
Mathematical Programming 188:135–192, 2021
arXiv slides video
Algorithms:
JacSketch
[73] Aritra Dutta, Xin Li and Peter Richtárik
Weighted low-rank approximation of matrices and background modeling
arXiv
Algorithms:
Weighted low-rank approximation of matrices and background modeling
arXiv
Algorithms:
WLR, inWLR
[72] Filip Hanzely and Peter Richtárik
Fastest rates for stochastic mirror descent methods
Computational Optimization and Applications 79:717–766, 2021
arXiv
Algorithms:
Fastest rates for stochastic mirror descent methods
Computational Optimization and Applications 79:717–766, 2021
arXiv
Algorithms:
relRCD, relSGD
[71] Lam M. Nguyen, Phuong Ha Nguyen, Marten van Dijk, Peter Richtárik, Katya Scheinberg and Martin Takáč
SGD and Hogwild! convergence without the bounded gradients assumption
Proceedings of The 35th International Conference on Machine Learning, PMLR 80:3750-3758, 2018
arXiv poster
Algorithms:
SGD and Hogwild! convergence without the bounded gradients assumption
Proceedings of The 35th International Conference on Machine Learning, PMLR 80:3750-3758, 2018
arXiv poster
Algorithms:
SGD, Hogwild!
[70] Robert M. Gower, Filip Hanzely, Peter Richtárik and Sebastian Stich
Accelerated stochastic matrix inversion: general theory and speeding up BFGS rules for faster second-order optimization
Advances in Neural Information Processing Systems 31:1619-1629, 2018
arXiv poster
Algorithms:
Accelerated stochastic matrix inversion: general theory and speeding up BFGS rules for faster second-order optimization
Advances in Neural Information Processing Systems 31:1619-1629, 2018
arXiv poster
Algorithms:
ABFGS
[69] Nikita Doikov and Peter Richtárik
Randomized block cubic Newton method
Proceedings of The 35th International Conference on Machine Learning, PMLR 80:1290-1298, 2018
Doikov: Best Talk Award, "Control, Information and Optimization", Voronovo, Russia, 2018
arXiv poster bib
Algorithms:
Randomized block cubic Newton method
Proceedings of The 35th International Conference on Machine Learning, PMLR 80:1290-1298, 2018
Doikov: Best Talk Award, "Control, Information and Optimization", Voronovo, Russia, 2018
arXiv poster bib
Algorithms:
RBCN
[68] Dmitry Kovalev, Eduard Gorbunov, Elnur Gasanov and Peter Richtárik
Stochastic spectral and conjugate descent methods
32nd Conference on Neural Information Processing Systems (NeurIPS 2018)
arXiv poster
Algorithms:
Stochastic spectral and conjugate descent methods
32nd Conference on Neural Information Processing Systems (NeurIPS 2018)
arXiv poster
Algorithms:
SSD, SconD, SSCD, mSSCD, iSconD, iSSD
[67] Radoslav Harman, Lenka Filová and Peter Richtárik
A randomized exchange algorithm for computing optimal approximate designs of experiments
Journal of the American Statistical Association, 2020
arXiv
Algorithms:
A randomized exchange algorithm for computing optimal approximate designs of experiments
Journal of the American Statistical Association, 2020
arXiv
Algorithms:
REX, OD_REX, MVEE_REX
[66] Ion Necoara, Andrei Patrascu and Peter Richtárik
Randomized projection methods for convex feasibility problems: conditioning and convergence rates
SIAM Journal on Optimization 29(4):2814–2852, 2019
arXiv slides
Algorithms:
Randomized projection methods for convex feasibility problems: conditioning and convergence rates
SIAM Journal on Optimization 29(4):2814–2852, 2019
arXiv slides
Algorithms:
SPA, SAP, AvPPrepared in 2017
[65] Nicolas Loizou and Peter Richtárik
Momentum and stochastic momentum for stochastic gradient, Newton, proximal point and subspace descent methods
Computational Optimization and Applications 77(3):653-710, 2020
arXiv
Algorithms:
Momentum and stochastic momentum for stochastic gradient, Newton, proximal point and subspace descent methods
Computational Optimization and Applications 77(3):653-710, 2020
arXiv
Algorithms:
mSGD, mSN, mSPP, mSDSA, smSGD, smSN, smSPP
[64] Aritra Dutta and Peter Richtárik
Online and batch supervised background estimation via L1 regression
IEEE Winter Conference on Applications in Computer Vision, 2019
arXiv
Algorithms:
Online and batch supervised background estimation via L1 regression
IEEE Winter Conference on Applications in Computer Vision, 2019
arXiv
Algorithms:
IRLS, Homotopy, SGD 1, SGD 2, ALM
[63] Nicolas Loizou and Peter Richtárik
Linearly convergent stochastic heavy ball method for minimizing generalization error
NIPS Workshop on Optimization for Machine Learning, 2017
arXiv poster
Algorithms:
Linearly convergent stochastic heavy ball method for minimizing generalization error
NIPS Workshop on Optimization for Machine Learning, 2017
arXiv poster
Algorithms:
SHB
[62] Dominik Csiba and Peter Richtárik
Global convergence of arbitrary-block gradient methods for generalized Polyak-Łojasiewicz functions
arXiv
Global convergence of arbitrary-block gradient methods for generalized Polyak-Łojasiewicz functions
arXiv
[61] Ademir Alves Ribeiro and Peter Richtárik
The complexity of primal-dual fixed point methods for ridge regression
Linear Algebra and its Applications 556:342-372, 2018
arXiv
Algorithms:
The complexity of primal-dual fixed point methods for ridge regression
Linear Algebra and its Applications 556:342-372, 2018
arXiv
Algorithms:
PDFP1, PDFP2, Quartz, New Quartz, Modified Quartz
[60] Matthias J. Ehrhardt, Pawel Markiewicz, Antonin Chambolle, Peter Richtárik, Jonathan Schott and Carola-Bibiane Schoenlieb
Faster PET reconstruction with a stochastic primal-dual hybrid gradient method
Proceedings of SPIE, Wavelets and Sparsity XVII, Volume 10394, pages 1039410-1 - 1039410-11, 2017
pdf video poster
Algorithms:
Faster PET reconstruction with a stochastic primal-dual hybrid gradient method
Proceedings of SPIE, Wavelets and Sparsity XVII, Volume 10394, pages 1039410-1 - 1039410-11, 2017
pdf video poster
Algorithms:
SPDHG
[59] Aritra Dutta, Xin Li and Peter Richtárik
A batch-incremental video background estimation model using weighted low-rank approximation of matrices
IEEE International Conference on Computer Vision (ICCV) Workshops, 2017
arXiv
Algorithms:
A batch-incremental video background estimation model using weighted low-rank approximation of matrices
IEEE International Conference on Computer Vision (ICCV) Workshops, 2017
arXiv
Algorithms:
inWLR
[58] Filip Hanzely, Jakub Konečný, Nicolas Loizou, Peter
Richtárik and Dmitry Grishchenko
Privacy preserving randomized gossip algorithms
arXiv slides
Algorithms:
Privacy preserving randomized gossip algorithms
arXiv slides
Algorithms:
Private Gossip with Binary Oracle, Private Gossip with ε-Gap Oracle, Private Gossip with Controlled Noise Insertion
[57] Antonin Chambolle, Matthias J. Ehrhardt, Peter Richtárik
and Carola-Bibiane Schoenlieb
Stochastic primal-dual hybrid gradient algorithm with arbitrary sampling and imaging applications
SIAM Journal on Optimization 28(4):2783-2808, 2018
arXiv slides video poster
Algorithms:
Stochastic primal-dual hybrid gradient algorithm with arbitrary sampling and imaging applications
SIAM Journal on Optimization 28(4):2783-2808, 2018
arXiv slides video poster
Algorithms:
SPDHG
[56] Peter Richtárik and Martin Takáč
Stochastic reformulations of linear systems: algorithms and convergence theory
SIAM Journal on Matrix Analysis and Applications 41(2):487–524, 2020
arXiv slides
Algorithms:
Stochastic reformulations of linear systems: algorithms and convergence theory
SIAM Journal on Matrix Analysis and Applications 41(2):487–524, 2020
arXiv slides
Algorithms:
basic, parallel and accelerated methods
[55] Mojmír Mutný and Peter Richtárik
Parallel stochastic Newton method
Journal of Computational Mathematics 36(3):404-425, 2018
arXiv
Algorithms:
Parallel stochastic Newton method
Journal of Computational Mathematics 36(3):404-425, 2018
arXiv
Algorithms:
PSNM
Prepared in 2016
[54] Robert M. Gower and Peter Richtárik
Linearly convergent randomized iterative methods for computing the pseudoinverse
arXiv
Algorithms:
Linearly convergent randomized iterative methods for computing the pseudoinverse
arXiv
Algorithms:
SATAX, SAXAS
[53] Jakub Konečný and Peter Richtárik
Randomized distributed mean estimation: accuracy vs communication
Frontiers in Applied Mathematics and Statistics 2018
arXiv
Algorithms:
Randomized distributed mean estimation: accuracy vs communication
Frontiers in Applied Mathematics and Statistics 2018
arXiv
Algorithms:
variable-size encoder, fixed-size encoder
[52] Jakub Konečný, H. Brendan McMahan, Felix Yu, Peter Richtárik, Ananda Theertha Suresh and Dave Bacon
Federated learning: strategies for improving communication efficiency
NIPS Private Multi-Party Machine Learning Workshop, 2016
link [selected press coverage: The Verge - Quartz - Vice CBR - Android Authority]
arXiv poster
Algorithms:
Federated learning: strategies for improving communication efficiency
NIPS Private Multi-Party Machine Learning Workshop, 2016
link [selected press coverage: The Verge - Quartz - Vice CBR - Android Authority]
arXiv poster
Algorithms:
structured updates, sketched updates
[51] Jakub Konečný, H. Brendan McMahan, Daniel Ramage and
Peter Richtárik
Federated optimization: distributed machine learning for on-device intelligence
link [selected press coverage: The Verge - Quartz - Vice CBR - Android Authority]
arXiv
Algorithms:
Federated optimization: distributed machine learning for on-device intelligence
link [selected press coverage: The Verge - Quartz - Vice CBR - Android Authority]
arXiv
Algorithms:
FSVRG
[50] Nicolas Loizou and Peter Richtárik
A new perspective on randomized gossip algorithms
IEEE Global Conference on Signal and Information Processing (GlobalSIP), 440-444, 2016
arXiv poster
Algorithms:
A new perspective on randomized gossip algorithms
IEEE Global Conference on Signal and Information Processing (GlobalSIP), 440-444, 2016
arXiv poster
Algorithms:
SDA, RBK, RNM
[49] Sashank J. Reddi, Jakub Konečný, Peter Richtárik, Barnabás Póczos, Alex Smola
AIDE: fast and communication efficient distributed optimization
arXiv poster
Algorithms:
AIDE: fast and communication efficient distributed optimization
arXiv poster
Algorithms:
Inexact DANE, AIDE
[48] Dominik Csiba and Peter Richtárik
Coordinate descent face-off: primal or dual?
Proceedings of Algorithmic Learning Theory, PMLR 83:246-267, 2018
arXiv bib
Algorithms:
Coordinate descent face-off: primal or dual?
Proceedings of Algorithmic Learning Theory, PMLR 83:246-267, 2018
arXiv bib
Algorithms:
NSync, QUARTZ
[47] Olivier Fercoq and Peter Richtárik
Optimization in high dimensions via accelerated, parallel and proximal coordinate descent
SIAM Review 58(4):739-771, 2016
SIAM SIGEST Award
arXiv
Algorithms:
Optimization in high dimensions via accelerated, parallel and proximal coordinate descent
SIAM Review 58(4):739-771, 2016
SIAM SIGEST Award
arXiv
Algorithms:
APPROX
[46] Robert M. Gower, Donald Goldfarb and Peter Richtárik
Stochastic block BFGS: squeezing more curvature out of data
Proceedings of the 33rd International Conference on Machine Learning, PMLR 48:1869-1878, 2016
arXiv poster bib
Algorithms:
Stochastic block BFGS: squeezing more curvature out of data
Proceedings of the 33rd International Conference on Machine Learning, PMLR 48:1869-1878, 2016
arXiv poster bib
Algorithms:
Stochastic Block BFGS
[45] Dominik Csiba and Peter Richtárik
Importance sampling for minibatches
Journal of Machine Learning Research 19(27):1-21, 2018
arXiv bib
Algorithms:
Importance sampling for minibatches
Journal of Machine Learning Research 19(27):1-21, 2018
arXiv bib
Algorithms:
dfSDCA
[44] Robert M. Gower and Peter Richtárik
Randomized quasi-Newton updates are linearly convergent matrix inversion algorithms
SIAM Journal on Matrix Analysis and Applications 38(4):1380-1409, 2017
Most Downloaded SIMAX Paper (6th place: 2018)
arXiv
Algorithms:
Randomized quasi-Newton updates are linearly convergent matrix inversion algorithms
SIAM Journal on Matrix Analysis and Applications 38(4):1380-1409, 2017
Most Downloaded SIMAX Paper (6th place: 2018)
arXiv
Algorithms:
SIMI, RBFGS, AdaRBFGS, ...
Prepared in 2015
[43] Zeyuan Allen-Zhu, Zheng Qu, Peter Richtárik and Yang Yuan
Even faster accelerated coordinate descent using non-uniform sampling
Proceedings of the 33rd International Conference on Machine Learning, PMLR 48:1110-1119, 2016
arXiv bib
Algorithms: NU_ACDM
[42] Robert M. Gower and Peter Richtárik
Stochastic dual ascent for solving linear systems
arXiv video
Algorithms: SDA
[41] Chenxin Ma, Jakub Konečný, Martin Jaggi, Virginia
Smith, Michael I Jordan, P. Richtárik and Martin Takáč
Distributed optimization with arbitrary local solvers
Optimization Methods and Software 32(4):813-848, 2017
Most-Read Paper, Optimization Methods and Software, 2017
arXiv
Algorithms: CoCoA+
[40] Martin Takáč, Peter Richtárik and Nathan Srebro
Distributed mini-batch SDCA
To appear in: Journal of Machine Learning Research
arXiv
Algorithms: mSDCA
[39] Robert M. Gower and Peter Richtárik
Randomized iterative methods for linear systems
SIAM
Journal on Matrix Analysis and Applications 36(4):1660-1690, 2015
Most Downloaded SIMAX Paper (1st place: 2017-2020)
Gower: 18th IMA Leslie Fox Prize (2nd Prize), 2017
link
arXiv
slides
Algorithms: sketch-and-project, GK, Gauss-LS, Gauss-pd
[38] Dominik Csiba and Peter Richtárik
Primal method for ERM with flexible mini-batching schemes and non-convex losses
arXiv
Algorithms: dfSDCA
[37] Jakub Konečný, Jie Liu, Peter Richtárik and Martin
Takáč
Mini-batch semi-stochastic gradient descent in the
proximal setting
IEEE
Journal of Selected Topics in Signal Processing 10(2): 242-255,
2016
arXiv
Algorithms: mS2GD
[36] Rachael Tappenden, Martin Takáč and Peter Richtárik
On the complexity of parallel coordinate descent
Optimization Methods and Software 33(2):372-395, 2018
arXiv
Algorithms: PCDM, PCDM-M
[35] Dominik Csiba, Zheng Qu and Peter Richtárik
Stochastic dual coordinate ascent with adaptive
probabilities
Proceedings
of the 32nd International Conference on Machine Learning,
PMLR 37:674-683, 2015
Csiba: Best Contribution Award (2nd
Place), Optimization and Big Data 2015
Implemented in Tensor Flow
arXiv poster bib
Algorithms: AdaSDCA and AdaSDCA+
[34] Chenxin Ma, Virginia Smith, Martin Jaggi, Michael I.
Jordan, Peter Richtárik and Martin Takáč
Adding vs. averaging in distributed primal-dual
optimization
Proceedings of the 32nd International Conference on Machine Learning,
PMLR 37:1973-1982, 2015
Smith: 2015 MLconf Industry Impact
Student Research Award link
CoCoA+ is now the default linear optimizer in Tensor Flow link
arXiv poster bib
Algorithms: CoCoA+
[33] Zheng Qu, Peter Richtárik, Martin Takáč and Olivier
Fercoq
SDNA: Stochastic dual Newton ascent for empirical risk
minimization
Proceedings
of the 33rd International Conference on Machine Learning, PMLR 48:1823-1832, 2016
arXiv slides poster bib
Algorithms: SDNA
Prepared in 2014
[32] Zheng Qu and Peter Richtárik
Coordinate descent with arbitrary sampling II: expected
separable overapproximation
Optimization
Methods and Software 31(5):858-884, 2016
arXiv
[31] Zheng Qu and Peter Richtárik
Coordinate descent with arbitrary sampling I: algorithms
and complexity
Optimization
Methods and Software 31(5):829-857, 2016
arXiv
Algorithms: ALPHA
[30] Jakub Konečný, Zheng Qu and Peter Richtárik
Semi-stochastic coordinate descent
Optimization
Methods and Software 32(5):993-1005, 2017
arXiv
Algorithms: S2CD
[29] Zheng Qu, Peter Richtárik and Tong Zhang
Quartz: Randomized dual coordinate ascent with arbitrary
sampling
Advances
in Neural Information Processing Systems 28:865-873, 2015
arXiv slides video
Algorithms: QUARTZ
[28] Jakub Konečný, Jie Liu, Peter Richtárik and Martin
Takáč
mS2GD: Mini-batch semi-stochastic gradient descent in the
proximal setting
NIPS
Workshop on Optimization for Machine Learning, 2014
arXiv poster
Algorithms: mS2GD
[27] Jakub Konečný, Zheng Qu and Peter Richtárik
S2CD: Semi-stochastic coordinate descent
NIPS
Workshop on Optimization for Machine Learning, 2014
pdf poster
Algorithms: S2CD
[26] Jakub Konečný and Peter Richtárik
Simple complexity analysis of simplified direct search
arXiv slides in Slovak
Algorithms: SDS
[25] Jakub Mareček, Peter Richtárik and Martin Takáč
Distributed block coordinate descent for minimizing
partially separable functions
Numerical
Analysis and Optimization, Springer Proceedings in Math. and
Statistics 134:261-288, 2015
arXiv
Algorithms: Distributed BCD
[24] Olivier Fercoq, Zheng Qu, Peter Richtárik and Martin
Takáč
Fast distributed coordinate descent for minimizing
non-strongly convex losses
2014
IEEE International Workshop on Machine Learning for Signal
Processing (MLSP), 2014
arXiv poster
Algorithms: Hydra^2
[23] Duncan Forgan and Peter Richtárik
On optimal solutions to planetesimal growth models
Technical Report ERGO 14-002, 2014
pdf
[22] Jakub Mareček, Peter Richtárik and Martin Takáč
Matrix completion under interval uncertainty
European
Journal of Operational Research 256(1):35-42, 2017
arXiv
Algorithms: MACO
Prepared in 2013
[21] Olivier Fercoq and Peter Richtárik
Accelerated, Parallel and PROXimal coordinate descent
SIAM
Journal on Optimization 25(4):1997-2023, 2015
Fercoq: 17th IMA Leslie Fox Prize
(Second Prize), 2015
2nd Most Downloaded SIOPT Paper (Aug
2016 - now)
arXiv video poster
Algorithms: APPROX
[20] Jakub Konečný and Peter Richtárik
Semi-stochastic gradient descent methods
Frontiers
in Applied Mathematics and Statistics 3:9, 2017
arXiv slides poster
Algorithms: S2GD and S2GD+
[19] Peter Richtárik and Martin Takáč
On optimal probabilities in stochastic coordinate descent
methods
Optimization Letters
10(6):1233-1243, 2016
arXiv poster
Algorithms: NSync
[18] Peter Richtárik and Martin Takáč
Distributed coordinate descent method for learning with
big data
Journal
of Machine Learning Research 17(75):1-25, 2016
arXiv poster
Algorithms: Hydra
[17] Olivier Fercoq and Peter Richtárik
Smooth minimization of nonsmooth functions with parallel
coordinate descent methods
Springer Proceedings in Mathematics and Statistics 279:57-96, 2019
arXiv
Algorithms: SPCDM
[16] Rachael Tappenden, Peter Richtárik and Burak Buke
Separable approximations and decomposition methods for
the augmented Lagrangian
Optimization
Methods and Software 30(3):643-668, 2015
arXiv
Algorithms: DQAM, PCDM
[15] Rachael Tappenden, Peter Richtárik and Jacek Gondzio
Inexact coordinate descent: complexity and
preconditioning
Journal
of Optimization Theory and Applications 170(1):144-176, 2016
arXiv poster
Algorithms: ICD
[14] Martin Takáč, Selin Damla Ahipasaoglu, Ngai-Man Cheung
and Peter Richtárik
TOP-SPIN: TOPic discovery via Sparse Principal component
INterference
Springer Proceedings in Mathematics and Statistics 279:157-180,
2019
arXiv poster
Algorithms: TOP-SPIN
[13] Martin Takáč, Avleen Bijral, Peter Richtárik and
Nathan Srebro
Mini-batch primal and dual methods for SVMs
Proceedings
of the 30th International Conference on Machine Learning,
2013
arXiv poster
Algorithms: minibatch SDCA and minibatch Pegasos
Prepared in 2012 or earlier
[12] Peter Richtárik, Majid Jahani, Martin Takáč and Selin Damla Ahipasaoglu
Alternating maximization: unifying framework for 8 sparse PCA formulations and efficient parallel codes
Optimization and Engineering 22:1493--1519, 2021
arXiv
Algorithms: 24am
[11] William Hulme, Peter Richtárik, Lynne McGuire and Alison Green
Optimal diagnostic tests for sporadic Creutzfeldt-Jakob disease based on SVM classification of RT-QuIC data
Technical Report, 2012
arXiv
[10] Peter Richtárik and Martin Takáč
Parallel coordinate descent methods for big data optimization
Mathematical Programming 156(1):433-484, 2016
Takáč: 16th IMA Leslie Fox Prize
(2nd Prize), 2013 link
#1 Top Trending Article in
Mathematical Programming Ser A and B (2017) link
arXiv slides
video
Algorithms: PCDM, AC/DC
[9] Peter Richtárik and Martin Takáč
Efficient serial and parallel coordinate descent methods for huge-scale truss topology design
Operations Research Proceedings 2011:27-32, Springer-Verlag, 2012
Optimization Online poster
Algorithms: Serial CD, Parallel CD
[8] Peter Richtárik and Martin Takáč
Iteration complexity of randomized block-coordinate descent methods for minimizing a composite function
Mathematical Programming 144(2):1-38, 2014
Best Student Paper (runner-up), INFORMS Computing Society, 2012
arXiv slides
Algorithms: RCDC, UCDC, RCDS
[7] Peter Richtárik and Martin Takáč
Efficiency of randomized coordinate descent methods on minimization problems with a composite objective function
Proceedings of Signal Processing with Adaptive Sparse
Structured Representations, 2011
pdf
Algorithms: UCDC
[6] Peter Richtárik
Finding sparse approximations to extreme eigenvectors: generalized power method for sparse PCA and extensions
Proceedings of
Signal Processing with Adaptive Sparse Structured Representations, 2011
pdf
Algorithms: GPower, ADM
[5] Peter Richtárik
Approximate level method for nonsmooth convex minimization
Journal
of Optimization Theory and Applications 152(2):334–350, 2012
Optimization Online
Algorithms: Approximate Level Method
[4] Michel Journée, Yurii Nesterov, Peter Richtárik and Rodolphe Sepulchre
Generalized power method for sparse principal component analysis
Journal of Machine Learning Research 11:517–553, 2010
arXiv slides poster
Algorithms: GPower
[3] Peter Richtárik
Improved algorithms for convex minimization in relative scale
SIAM Journal on Optimization 21(3):1141–1167, 2011
pdf slides
Algorithms: SubBis, SubSearchNR, SubBisNR, SmoothBis
[2] Peter Richtárik
Simultaneously solving seven optimization problems in relative scale
Technical Report, 2009
Optimization Online
Algorithms: Inc, IncDec
[1] Peter Richtárik
Some algorithms for large-scale convex and linear minimization in relative scale
PhD Dissertation, School of Operations Research and Information Engineering, Cornell University, 2007
Algorithms: SubBis, SmoothBis, Inc, IncDec