| Compressed L2GD | Personalized federated learning with communication compression. Based on the paper Personalized Federated Learning with Communication Compression. |
| EF21 | Error feedback experiments for EF21 and its practical extensions (EF21-SGD, EF21-PAGE, EF21-PP, EF21-BC, EF21-HB, EF21-Prox). Based on the papers EF21: A New, Simpler, Theoretically Better, and Practically Faster Error Feedback and EF21 with Bells & Whistles. |
| RR, SO | Random Reshuffling and Shuffle Once algorithms. Based on the paper Random Reshuffling: Simple Analysis with Vast Improvements. |
| SAGA-AS | SAGA with arbitrary sampling. Based on this ICML 2019 paper. |
| REX | Randomized EXchange algorithms, implemented as web-based R Shiny apps, for computing optimal design of experiments and minimum volume enclosing ellipsoids. Based on this paper. |
| ABPG, ABDA | Accelerated Bregman proximal gradient methods for relatively smooth convex optimization. Based on this paper. |
| JacSketch | Stochastic quasi-gradient methods with variance reduction via Jacobian sketching. Based on this paper. |
| SPDHG | Stochastic Chambolle-Pock method (Stochastic Primal-Dual Hybrid Gradient). Based on this paper. See also this follow-up paper with application to PET imaging. |
| StochBFGS | Stochastic (block) BFGS method for empirical risk minimization with logistic loss and L2 regularizer. Related paper. |
| Random Inverse | A suite of randomized methods for inverting positive definite matrices, implemented in MATLAB. Related paper. |
| Random Linear Lab | A lab for testing and comparing randomized methods for solving linear systems, implemented in MATLAB. Related paper. |
| CoCoA | A framework for communication-efficient distributed optimization for machine learning. |
| APPROX | Accelerated, Parallel and PROXimal coordinate descent. Efficient C++ code based on this paper, also implementing PCDM, SDCA, and AGD. Also available as an accelerated option in scikit-learn for LASSO / elastic net. |
| S2GD | Semi-stochastic gradient descent for fast training of L2-regularized logistic regression. Efficient C++ code (can be called from MATLAB), based on this paper. |
| 24am | Parallel Sparse PCA (8, 9). Supports multicore workstations, GPUs, and clusters. The cluster version was tested on terabyte matrices. Extension of GPower. |
| AC-DC | Serial (1, 5), parallel (2, 3, 4), and distributed (6, 7) coordinate descent for big data optimization. The parallel and distributed codes can solve LASSO instances with terabyte matrices and billions of features. |
Professor of Computer Science
King Abdullah University of Science and Technology
Office 3145, Bldg 12, 4700 KAUST, Thuwal 23955-6900, Saudi Arabia
