Aritra Dutta
|
|
News
December 19th 2022
Our paper On the Convergence Analysis of Asynchronous SGD for Solving Consistent Linear Systems got accepted for publication in Elsevier's Linear Algebra and its Applications. This a joint work with Atal Narayan Sahu, Aasutosh Tiwari, and Peter Richtárik from KAUST.
[During the review, we made considerable changes in the manuscript and it differs from the original ArXiv version of the manuscript. We plan to update the ArXiv version shortly. Until then, here is a preprint version of the manuscript. ]
October 3rd 2022
Our paper Direct Nonlinear Acceleration got accepted for publication in Elsevier's EURO Journal on Computational Optimization. This a joint work with Elhoucine Bergou of UM6P, Marco Canini and Peter Richtárik from KAUST, Yunming Xiao of Northwestern University.
September 12th 2022
New Paper out: Personalized Federated Learning with Communication Compression, joint work with Elhoucine Bergou of UM6P, Konstantin Burlachenko and Peter Richtárik of KAUST
Abstract: In contrast to training traditional machine learning (ML) models in data centers, federated learning (FL) trains ML models over local datasets contained on resource-constrained heterogeneous edge devices. Existing FL algorithms aim to learn a single global model for all participating devices, which may not be helpful to all devices participating in the training due to the heterogeneity of the data across the devices. Recently, Hanzely and Richtárik (2020) proposed a new formulation for training personalized FL models aimed at balancing the trade-off between the traditional global model and the local models that could be trained by individual devices using their private data only. They derived a new algorithm, called Loopless Gradient Descent (L2GD), to solve it and showed that this algorithms leads to improved communication complexity guarantees in regimes when more personalization is required. In this paper, we equip their L2GD algorithm with a bidirectional compression mechanism to further reduce the communication bottleneck between the local devices and the server. Unlike other compression-based algorithms used in the FL-setting, our compressed L2GD algorithm operates on a probabilistic communication protocol, where communication does not happen on a fixed schedule. Moreover, our compressed L2GD algorithm maintains a similar convergence rate as vanilla SGD without compression. To empirically validate the efficiency of our algorithm, we perform diverse numerical experiments on both convex and non-convex problems and using various compression techniques.
May 18th 2022
I am delighted and honored to join the Pioneer Centre for AI (P1) as an affiliated researcher. Quoting from the under-constructed website of the Pioneer Center of AI: Hosted by the University of Copenhagen’s Department of Computer Science, with cooperating institutions, Denmark’s Technical University, IT University of Copenhagen, Aalborg University, and Aarhus University, and co-lead by Director, Professor Serge Belongie, the Pioneer Centre for AI (P1) focuses on fundamental research, and within an interdisciplinary framework, develops platforms, methods, and practices addressing society’s greatest challenges, and one of its kind.
May 12th 2022
I am presently visiting Professor Xin Li at the United States for a summer research visit. I will spend the entire summer working with him, and then finally I will attend the seventh International Conference on Continuous Optimization (ICCOPT)---one of the flagship venues for Optimization held in every three years. At ICCOPT, Prof. Elhoucine Bergou and I are organizing a session titled, "Stochastic optimization methods in Machine Learning" (covering topics from distributed computing, stochastic and deterministic optimization, algorithmic aspects for learning, approximation theory, and signal processing, etc.) at the Nonlinear Optimization Cluster. Additionally, I will give an invited talk in a session organized by Prof. Youssef Diouane, Prof. Serge Gratton, and Prof. Elhoucine Bergou.
September 29th 2021
Today is my last official working day at KAUST (King Abdullah University of Science and Technology), and the script can not be written any better. Special thanks to Panos Kalnis, Peter Richtárik, and my PhD adviser Xin Li for their immense support in this journey. Aritra out!
September 28th 2021
We got two papers accepted in Thirty-fifth Conference on Neural Information Processing Systems (NeuRIPS 2021)---Ranked 12 by Google Scholar among all conferences and journals across all fields.
July 19th 2021
I received the Best Reviewer Award (Top 10%) from the International Conference on Machine Learning 2021 (ICML 2021).
July 3rd 2021
I will start a new position as an Assistant Professor in the Department of Mathematics and Computer Science at the University of Southern Denmark (Syddansk Universitet, SDU) beginning October 2021.
May 15th 2021
I am organizing a special session at the INFORMS annual meeting, to be held in Anaheim, California, USA, October 24-27th 2021, jointly with El Houcine Bergou. The title of the session is "Stochastic Optimization Methods in Machine Learning". We have a wonderful list of speakers.
March 19th 2021
I received the "Outstanding Review Award" from International Conference on Learning Representation, 2021.
March 17th 2021
Our paper GRACE: A Compressed Communication Framework for Distributed Machine Learning got accepted in 41st IEEE International Conference on Distributed Computing Systems (ICDCS). For the full technical version of the paper please see Compressed Communication for Distributed Deep Learning: Survey and Quantitative Evaluation. The codebase is available here codebase-GRACE.
According to the ICDCS acceptance email "The selection process was very competitive this year" yielding an acceptance rate of 19.8 %.
February 4th 2021
I accepted an invitation to join the Editorial Board of Signal Processing Theory as Review Editor for Frontiers in Signal Processing.
February 8th 2021
New Paper out: DeepReduce: A Sparse-tensor Communication Framework for Distributed Deep Learning, joint work with Kelly Kostopoulou from Columbia University, Xin Li from University of Central Florida, Hang Xu and Panos Kalnis from KAUST, and Alexandros Ntoulas from National and Kapodistrian University of Athens.
Abstract: Sparse tensors appear frequently in distributed deep learning, either as a direct artifact of the deep neural network's gradients, or as a result of an explicit sparsification process. Existing communication primitives are agnostic to the peculiarities of deep learning; consequently, they impose unnecessary communication overhead. This paper introduces DeepReduce, a versatile framework for the compressed communication of sparse tensors, tailored for distributed deep learning. DeepReduce decomposes sparse tensors in two sets, values and indices, and allows both independent and combined compression of these sets. We support a variety of common compressors, such as Deflate for values, or run-length encoding for indices. We also propose two novel compression schemes that achieve superior results: curve fitting-based for values and bloom filter-based for indices. DeepReduce is orthogonal to existing gradient sparsifiers and can be applied in conjunction with them, transparently to the end-user, to significantly lower the communication overhead. As proof of concept, we implement our approach on Tensorflow and PyTorch. Our experiments with large real models demonstrate that DeepReduce transmits fewer data and imposes lower computational overhead than existing methods, without affecting the training accuracy.
January 21st 2021
New Paper out: A Fast and Adaptive SVD-free Algorithm for General Weighted Low-rank Recovery, joint work with Jingwei Liang of University of Cambridge, and Xin Li of University of Central Florida.
Abstract: This paper is devoted to proposing a general weighted low-rank recovery model and designs a fast SVD-free computational scheme to solve it. First, our generic weighted low-rank recovery model unifies several existing approaches in the literature. Moreover, our model readily extends to the non-convex setting. Algorithm-wise, most first-order proximal algorithms in the literature for low-rank recoveries, require computing singular value decomposition (SVD). As SVD does not scale properly with the dimension of the matrices, these algorithms becomes slower when the problem size becomes larger. By incorporating the variational formulation of the nuclear norm into the sub-problem of proximal gradient descent, we avoid computing SVD which results in significant speed-up. Moreover, our algorithm preserves the rank identification property of nuclear norm [33] which further allows us to design a rank continuation scheme that asymptotically achieves the minimal iteration complexity. Numerical experiments on both toy example and real-world problems including structure from motion (SfM) and photometric stereo, background estimation and matrix completion, demonstrate the superiority of our proposed algorithm.
January 12th 2021
I have accepted an invitation to serve as a Reviewer of International Conference on Computer Vision 2021 (ICCV 2021).
December 19th 2020
I have accepted an invitation to serve as a Reviewer of International Conference on Machine Learning 2021 (ICML 2021).
October 12th 2020
The public GITHUB repository called Distributed ML for our paper Huffman Coding Based Encoding Techniques for Fast Distributed Deep Learning is available. This repository provides essential encoding techniques used in our work. Any questions/comments are welcome.
November 9th 2020
I have accepted an invitation to serve as a Session Chair of the Machine Learning Session at the Twenty-Seventh International Conference on Neural Information Processing (ICONIP 2020).
October 12th 2020
Our paper Huffman Coding Based Encoding Techniques for Fast Distributed Deep Learning, joint work with Rishikesh R. Gajjala, Shashwat Banchhor of IIT Delhi and Ahmed Sayed, Marco Canini, and Panos Kalnis of KAUST got accepted for publication in ACM 16th International Conference on emerging Networking EXperiments and Technologies (CoNEXT), DistributeML workshop.
September 2020
I have accepted invitations to serve as a reviewer for the International Conference on Learning Representations (ICLR 2021), IEEE Computer Vision and Pattern Recognition 2021 (CVPR 2021), and as a Program Committee (PC) member of Thirty-Fifth AAAI Conference on Artificial Intelligence (AAAI-21).
July 7th 2020
Our paper Best Pair Formulation & Accelerated Scheme for Non-convex Principal Component Pursuit, joint work with Jingwei Liang of University of Cambridge, Filip Hanzley of KAUST, and Peter Richtárik of KAUST got accepted for publication as regular paper in the IEEE Transactions on Signal Processing (2018-'19 Impact Factor 6.083).
June 7th 2020
I have accepted an invitation to serve as a Program Committee (PC) member for the IEEE CVF Winter Conference on Applications of Computer Vision (WACV 2021).
May 8th 2020
I have accepted an invitation to serve as a Program Committee (PC) member for the Twenty-Seventh International Conference on Neural Information Processing (ICONIP 2020).
May 7th 2020
We have made our codebase-GRACE(GRAdient ComprEssion for distributed deep learning) public for the paper Compressed Communication for Distributed Deep Learning: Survey and Quantitative Evaluation. Any feedback and/or ideas for future collaboration around this topic is welcome!
April 13th 2020
New Paper out: Compressed Communication for Distributed Deep Learning: Survey and Quantitative Evaluation joint work with Hang Xu, Chen-Yu Ho, Ahmed Mohamed Abdelmoniem Sayed, El Houcine Bergou, Konstantinos Karatsenidis, Marco Canini, and Panos Kalnis of KAUST.
Abstract: Powerful computer clusters are used nowadays to train complex deep neural networks (DNN) on large datasets. Distributed training workloads increasingly become communication bound. For this reason, many lossy compression techniques have been proposed to reduce the volume of transferred data. Unfortunately, it is difficult to argue about the behavior of compression methods, because existing work relies on inconsistent evaluation testbeds and largely ignores the performance impact of practical system configurations. In this paper, we present a comprehensive survey of the most influential compressed communication methods for DNN training, together with an intuitive classification (i.e., quantization, sparsification, hybrid and low-rank). We also propose a unified framework and API that allows for consistent and easy implementation of compressed communication on popular machine learning toolkits. We instantiate our API on TensorFlow and PyTorch, and implement 16 such methods. Finally, we present a thorough quantitative evaluation with a variety of DNNs (convolutional and recurrent), datasets and system configurations. We show that the DNN architecture affects the relative performance among methods. Interestingly, depending on the underlying communication library and computational cost of compression/decompression, we demonstrate that some methods may be impractical.
April 7th 2020
New Paper out: On the Convergence Analysis of Asynchronous SGD for Solving Consistent Linear Systems joint work with Atal Narayan Sahu of KAUST, Aashutosh Tiwari of IIT, Kanpur, and Peter Richtárik of KAUST and MIPT.
Abstract: In the realm of big data and machine learning, data-parallel, distributed stochastic algorithms have drawn significant attention in the present days. While the synchronous versions of these algorithms are well understood in terms of their convergence, the convergence analyses of their asynchronous counterparts are not widely studied. In this paper, we propose and analyze a distributed, asynchronous parallel SGD in light of solving an arbitrary consistent linear system by reformulating the system into a stochastic optimization problem as studied by Richtárik and Takáč in [35]. We compare the convergence rates of our asynchronous SGD algorithm with the synchronous parallel algorithm proposed by Richtárik and Takáč in [35] under different choices of the hyperparameters---the stepsize, the damping factor, the number of processors, and the delay factor. We show that our asynchronous parallel SGD algorithm also enjoys a global linear convergence rate, similar to the basic method and the synchronous parallel method in [35] for solving any arbitrary consistent linear system via stochastic reformulation. We also show that our asynchronous parallel SGD improves upon the basic method with a better convergence rate when the number of processors is larger than four. We further show that this asynchronous approach performs asymptotically better than its synchronous counterpart for certain linear systems. Moreover, for certain linear systems, we compute the minimum number of processors required for which our asynchronous parallel SGD is better, and find that this number can be as low as two for some ill-conditioned problems.
March 23rd 2020
I have accepted an invitation to serve as a reviewer for the Thirty-fourth Conference on Neural Information Processing Systems 2020.
March 20th 2020
The website for 29th International Joint Conference on Artificial Intelligence and the 17th Pacific Rim International Conference on Artificial Intelligence, IJCAI-PRICAI 2020 tutorial proposal Compressed Communication for Large-scale Distributed Deep Learning is up and running.
March 6th 2020
Our tutorial proposal Compressed Communication for Large-scale Distributed Deep Learning jointly with El Houcine Bergou and Prof. Panos Kalnis got accepted in the 29th International Joint Conference on Artificial Intelligence and the 17th Pacific Rim International Conference on Artificial Intelligence, IJCAI-PRICAI 2020. The conference will be held in July 11-17th, 2020 in Yokohama, Japan. For the final schedule please visit IJCAI-PRICAI 2020. Stay tuned with the tutorial webpage and the online material. For any queries and details regarding the tutorial please contact me or Dr Bergou.
February 5th-13th 2020
I will be attending the Thirty-Fourth AAAI Conference on Artificial Intelligence (AAAI-20) from February 5th till February 13th, 2020. The conference will be held at the Hilton New York Midtown. I will present my joint work accepted and designated for spotlight and poster presentation at AAAI 2020 on Thursday, February 11th at the 55th Session from 11:15 am-12:30 pm, Paper ID 2100, titled: On the Discrepancy between the Theoretical Analysis and Practical Implementations of Compressed Communication for Distributed Deep Learning We will present in the poster session on the evening of the same day (11th February) as our poster spotlight talk. Feel free to get in touch to discuss about any future collaboration or to just grab a coffee.
December 2019
I am organizing a special session at the SIAM conference on Optimization 2020 (SIOPT 2020) to be held at Hong Kong in May 2020. SIOPT is a flagship conference series on mathematical optimization, covering a wide range of topics in the field. For more details about the conference visit this webpage.
November 20th 2019
I have accepted an invitation to serve as a reviewer for the International Conference on Machine Learning 2020 (ICML 2020).
November 19th 2019
New Paper out: On the Discrepancy between the Theoretical Analysis and Practical Implementations of Compressed Communication for Distributed Deep Learning joint work with El Houcine Bergou, Ahmed Mohamed Abdelmoniem Sayed, Chen-Yu Ho, Atal Narayan Sahu , Marco Canini, and Panos Kalnis.
Venue: To appear in Thirty-Fourth AAAI Conference on Artificial Intelligence (AAAI-20).
Abstract: Compressed communication, in the form of sparsification or quantization of stochastic gradients, is employed to reduce communication costs in distributed data-parallel training of deep neural networks. However, there exists a discrepancy between theory and practice: while theoretical analysis of most existing compression methods assumes compression is applied to the gradients of the entire model, many practical implementations operate individually on the gradients of each layer of the model.
In this paper, we prove that layer-wise compression is, in theory, better, because the convergence rate is upper bounded by that of entire-model compression for a wide range of biased and unbiased compression methods. However, despite the theoretical bound, our experimental study of six well-known methods shows that convergence, in practice, may or may not be better, depending on the actual trained model and compression ratio. Our findings suggest that it would be advantageous for deep learning frameworks to include support for both layer-wise and entire-model compression.
November 11th 2019
Our paper On the Discrepancy between the Theoretical Analysis and Practical Implementations of Compressed Communication for Distributed Deep Learning, joint work with El Houcine Bergou, Ahmed M. Ab, Chen-Yu Ho, Atal Narayan Sahu, Marco Canini, and Panos Kalnis, got accepted in Thirty-Fourth AAAI Conference on Artificial Intelligence (AAAI-20).
According to the AAAI acceptance email there was a record number of over 8,800 submissions this year. Of those, only 1,591 papers got accepted, yielding an acceptance rate of 20.6%.
September 18th 2019
I have accepted an invitation to serve as a reviewer for the IEEE Computer Vision and Pattern Recognition 2020 (CVPR 2020).
August 16th 2019
I have accepted an invitation to serve as a Program Committee (PC) member for the Thirty-Fourth AAAI Conference on Artificial Intelligence (AAAI-20).
August 2nd-9th 2019
I am now in Berlin, Germany, organizing three special sessions at the Sixth International Conference on Continuous Optimization 2019, jointly with El Houcine Bergou. Title of the sessions are: "New Trends in Optimization Methods and Machine Learning I, II, and III".
June 6th 2019
I will be attending the Thirty-sixth International Conference on Machine Learning (ICML) from June 10th till 15th, 2019 to be held at the Long Beach Convention Center, Long Beach, Los Angeles, California. Our group will present several papers. Feel free to get in touch to discuss any future collaboration or to just grab a coffee.
May 26th 2019
New Paper out: Direct Nonlinear Acceleration, joint work with El Houcine Bergou of KAUST and INRA, Marco Canini of KAUST, Yunming Xiao of Tsinghua University, and Peter Richtárik of KAUST and MIPT.
Abstract: Optimization acceleration techniques such as momentum play a key role in state-of-the-art machine learning algorithms. Recently, generic vector sequence extrapolation techniques, such as regularized nonlinear acceleration (RNA) of Scieur et al., were proposed and shown to accelerate fixed point iterations. In contrast to RNA which computes extrapolation coefficients by (approximately) setting the gradient of the objective function to zero at the extrapolated point, we propose a more direct approach, which we call direct nonlinear acceleration (DNA). In DNA, we aim to minimize (an approximation of) the function value at the extrapolated point instead. We adopt a regularized approach with regularizers designed to prevent the model from entering a region in which the functional approximation is less precise. While the computational cost of DNA is comparable to that of RNA, our direct approach significantly outperforms RNA on both synthetic and real-world datasets. While the focus of this paper is on convex problems, we obtain very encouraging results in accelerating the training of neural networks.
May 26th 2019
New Paper out: Best Pair Formulation & Accelerated Scheme for Non-convex Principal Component Pursuit, joint work with Jingwei Liang of University of Cambridge, Filip Hanzley of KAUST, and Peter Richtárik of KAUST and MIPT.
Abstract: The best pair problem aims to find a pair of points that minimize the distance between two disjoint sets. In this paper, we formulate the classical robust principal component analysis (RPCA) as the best pair; which was not considered before. We design an accelerated proximal gradient scheme to solve it, for which we show global convergence, as well as the local linear rate. Our extensive numerical experiments on both real and synthetic data suggest that the algorithm outperforms relevant baseline algorithms in the literature.
March 12th 2018
I have accepted an invitation to serve as a reviewer for the Thirty-third Conference on Neural Information Processing Systems 2019.
March 8th 2019
I am organizing two special sessions at the Sixth International Conference on Continuous Optimization 2019, to be held in Berlin, Germany, August 3-8th 2019, jointly with El Houcine Bergou. We will start sending out the invitations soon. Title of the session is "New Trends in Optimization Methods and Machine Learning I and II".
ICCOPT is a flagship conference series of the Mathematical Optimization Society (MOS) on continuous optimization, covering a wide range of topics in the field. Our sessions are part of the “Non-Linear Optimization” cluster.
January 24th 2019
I will be attending the Thirty-Third AAAI Conference on Artificial Intelligence from January 27th till February 1st, 2019. The conference will be held at the Hilton Hawaiian Village, Honolulu, Hawaii. I will present my joint work with Filip Hanzley and Peter Richtárik accepted and designated for oral presentation at AAAI 2019 on January 30th - Paper ID 5833 titled: A Nonconvex Projection Method for Robust PCA. We will present in the poster session as well. Feel free to get in touch to discuss about any future collaboration or to just grab a coffee. Let's get the party started as ICML deadline is over.
January 13th 2019
I am back to KAUST and working on ICML deadline. I am unable to join American Mathematical Society's Joint Mathematics Meeting 2019 to be held in Baltimore, USA, January 16-19, 2019 due to extremely busy travel and research schedule in this month (two Hawaii trips and ICML deadline!!). But our session "Optimal Methods in Applicable Analysis: Variational Inequalities, Low Rank Matrix Approximations, Systems Engineering, Cyber Security" (SS-81) is going to ROCK! I thank all the special session invitees and apologize to my old friends for not catching up with you in JMM this year.
January 5th 2019
I will be attending the WACV 2019 – IEEE Winter Conference on the Applications of Computer Vision from January 7th till January 10th, 2019. The conference will be held at the Hilton Waikoloa Village, Hawaii. I will present my joint work with Peter Richtárik accepted at WACV- Online and Batch Supervised Background Estimation via L1 Regression (paper ID 84) as a 5 minutes spotlight talk in the oral Session 2-A titled "Visions Systems and Applications" (on Tuesday, 8th January 2019, 3:20-5:00pm) followed by a two hour poster session. I am on my way to Hilton Waikoloa Village. Feel free to get in touch to talk about my present work and for future collaboration.
December 14th 2018
Our paper A Nonconvex Projection Method for Robust PCA is designated for oral presentation at the Thirty-Third AAAI Conference on Artificial Intelligence (AAAI-19). The email says : "papers were chosen for the conference according to a uniform standard of technical contribution. Selection of presentation format for accepted papers was based primarily on an assessment of breadth of interest, and the construction of balanced and topically coherent sessions."
November 14th 2018
Finally, I am back to KAUST after attending INFORMS and spending my vacation. I presented a poster based on our recent work A Nonconvex Projection Method for Robust PCA at the Statistics and Data Science workshop at KAUST.
November 4th 2018
Our paper Online and Batch Supervised Background Estimation via L1 Regression got accepted in WACV 2019 – IEEE Winter Conference on the Applications of Computer Vision. It was a joint work with Peter Richtárik of KAUST, MIPT, and University of Edinburgh. The conference will take place in Waikoloa Village, Hawaii, USA, January 7-11, 2019.
November 1st 2018
I am attending the 2018 INFORMS Annual Meeting. I am also chairing two sessions, jointly with Dr El Houcine Bergou. Title of our sessions are: "Stochastic Optimization Methods and Approximation Theory in Machine Learning I and II".
October 31st 2018
Our paper A Nonconvex Projection Method for Robust PCA, joint work with Filip Hanzley of KAUST and Peter Richtárik of KAUST, MIPT, University of Edinburgh got accepted in Thirty-Third AAAI Conference on Artificial Intelligence (AAAI-19).
According to the AAAI acceptance email there was a record number of over 7,700 submissions this year. Of those, 7,095 were reviewed, and only 1,150 papers got accepted, yielding an acceptance rate of 16.2%. The email quotes "There was especially stiff competition this year because of the number of submissions, and you should be proud of your success. " The conference will take place in Honolulu, Hawaii, USA, January 27-February 1, 2019.
October 19th 2018
I have accepted an invitation to serve as a reviewer for The 36th International Conference on Machine Learning (ICML 2019) which is to be held in Long Beach, California, June 10-15, 2019. I am looking forward to learn many exciting things by reviewing some cutting edge work. Paper submission deadline: January 23, 2019
June 4th-9th 2018
I am attending SIAM Conference on Imaging Science 2018 at Bologna, Italy. I will give a talk at the minisymposia Computational Methods for Large-Scale Machine Learning in Imaging. It is an amazing place and the conference is great so far. I am excited to see some of my old friends and collaborators.
May 19th 2018
New Paper out: A Nonconvex Projection Method for Robust PCA, joint work with Filip Hanzley of KAUST and Peter Richtárik of KAUST, MIPT, University of Edinburgh.
Abstract: Robust principal component analysis (RPCA) is a well-studied problem with the goal of decomposing a matrix into the sum of low-rank and sparse components. In this paper, we propose a nonconvex feasibility reformulation of RPCA problem and apply an alternating projection method to solve it. To the best of our knowledge, we are the first to propose a method that solves RPCA problem without considering any objective function, convex relaxation, or surrogate convex constraints. We demonstrate through extensive numerical experiments on a variety of applications, including shadow removal, background estimation, face detection, and galaxy evolution, that our approach matches and often significantly outperforms current state-of-the-art in various ways.
May 12th 2018
We are overwhelmed by the extraordinary response from the invitees. Now we have two special sessions at 2018 INFORMS annual meeting: "Stochastic Optimization Methods and Approximation Theory in Machine Learning-I and II". Dr Bergou and I are thankful to those who participated.
May 9th 2018
I am organizing a special session jointly with Prof. Ram N. Mohapatra at the American Mathematical Society's Joint Mathematics Meeting 2019 to be held in Baltimore, USA, January 16-19, 2019. The title of the special session is "Optimal Methods in Applicable Analysis: Variational Inequalities, Low Rank Matrix Approximations, Systems Engineering, Cyber Security" (SS 81). We are grateful to the AMS for granting us a total 7 hours for our session. JMM is world's largest mathematics meeting with 6400+ participants in last year's annual meeting. It is going to be very exciting. We are going to send out the invitations soon. I also thank Dr Mohapatra for working on the proposal with me.
May 4th 2018
Our special session "Stochastic Optimization Methods and Approximation Theory in Machine Learning" to be held at INFORMS 2018 annual meeting has five extraordinary speakers. Dr Bergou and I are thankful to those who participated. The list of the speakers is coming soon! I am looking for two more speakers which can result in to another session. If anyone is interested in participating please contact me.
May 3rd 2018
I graduated with the Udacity Deep Learning Nanodegree! I learned some great stuff and did some exciting projects. Looking forward to doing some hands on work in Deep Learning.
April 18th 2018
I am organizing a special session at the INFORMS annual meeting, to be held in Phoenix, Arizona, November 4-7th 2018, jointly with El Houcine Bergou. We will start sending out the invitations soon. Title of the session is "Stochastic Optimization Methods and Approximation Theory in Machine Learning".
April 15th 2018
New Paper out: Weighted Low-Rank Approximation for Background Modeling, joint work with Xin Li of UCF and Peter Richtárik of KAUST.
Abstract: We primarily study a special a weighted low-rank approximation of matrices and then apply it to solve the background modeling problem. We propose two algorithms for this purpose: one operates in the batch mode on the entire data and the other one operates in the batch-incremental mode on the data and naturally captures more background variations and computationally more effective. Moreover, we propose a robust technique that learns the background frame indices from the data and does not require any training frames. We demonstrate through extensive experiments that by inserting a simple weight in the Frobenius norm, it can be made robust to the outliers similar to the L1 norm. Our methods match or outperform several state-of-the-art online and batch background modeling methods in virtually all quantitative and qualitative measures.
March 12th 2018
I went to attend the workshop Integrating Machine Learning and Predictive Simulation: From Uncertainty Quantification to Digital Twins held in Institute for Mathematics and its Applications (IMA), 4-8th March, 2018. Thanks to the organizers and generous funding of IMA for providing me with a comprehensive travel grant. I had a great time at Minneapolis, Minnesota.
March 9th 2018
I gave an alumni talk at the Mathematics department, University of Central Florida.
May 2017
I was fortunate to be featured as one of the successful graduate students on the University of Central Florida's Mathematics department's doctoral program's brochure. It has been a great honor. I was also fortunate to be featured on the University of Central Florida's Graduate Catalog.
April 2017
I was selected as a winner of Lee H. Armstrong award for excellence in Graduate teaching, 2017.
16th December 2016
I graduated with a PhD in Mathematics. Thanks to my advisors: Prof. Xin Li and Prof. Qiyu Sun. I also thank my other committee members: Prof. Mubarak Shah, Prof. Ram N Mohapatra, and Prof. Zuhair Nashed. I also thank my dearest friends and family members. Without their support and love nothing would have been possible.
November 2016
I won the Outstanding Dissertation Award 2016 from the Mathematics Department, University of Central Florida.
Our paper On the Convergence Analysis of Asynchronous SGD for Solving Consistent Linear Systems got accepted for publication in Elsevier's Linear Algebra and its Applications. This a joint work with Atal Narayan Sahu, Aasutosh Tiwari, and Peter Richtárik from KAUST.
[During the review, we made considerable changes in the manuscript and it differs from the original ArXiv version of the manuscript. We plan to update the ArXiv version shortly. Until then, here is a preprint version of the manuscript. ]
October 3rd 2022
Our paper Direct Nonlinear Acceleration got accepted for publication in Elsevier's EURO Journal on Computational Optimization. This a joint work with Elhoucine Bergou of UM6P, Marco Canini and Peter Richtárik from KAUST, Yunming Xiao of Northwestern University.
September 12th 2022
New Paper out: Personalized Federated Learning with Communication Compression, joint work with Elhoucine Bergou of UM6P, Konstantin Burlachenko and Peter Richtárik of KAUST
Abstract: In contrast to training traditional machine learning (ML) models in data centers, federated learning (FL) trains ML models over local datasets contained on resource-constrained heterogeneous edge devices. Existing FL algorithms aim to learn a single global model for all participating devices, which may not be helpful to all devices participating in the training due to the heterogeneity of the data across the devices. Recently, Hanzely and Richtárik (2020) proposed a new formulation for training personalized FL models aimed at balancing the trade-off between the traditional global model and the local models that could be trained by individual devices using their private data only. They derived a new algorithm, called Loopless Gradient Descent (L2GD), to solve it and showed that this algorithms leads to improved communication complexity guarantees in regimes when more personalization is required. In this paper, we equip their L2GD algorithm with a bidirectional compression mechanism to further reduce the communication bottleneck between the local devices and the server. Unlike other compression-based algorithms used in the FL-setting, our compressed L2GD algorithm operates on a probabilistic communication protocol, where communication does not happen on a fixed schedule. Moreover, our compressed L2GD algorithm maintains a similar convergence rate as vanilla SGD without compression. To empirically validate the efficiency of our algorithm, we perform diverse numerical experiments on both convex and non-convex problems and using various compression techniques.
May 18th 2022
I am delighted and honored to join the Pioneer Centre for AI (P1) as an affiliated researcher. Quoting from the under-constructed website of the Pioneer Center of AI: Hosted by the University of Copenhagen’s Department of Computer Science, with cooperating institutions, Denmark’s Technical University, IT University of Copenhagen, Aalborg University, and Aarhus University, and co-lead by Director, Professor Serge Belongie, the Pioneer Centre for AI (P1) focuses on fundamental research, and within an interdisciplinary framework, develops platforms, methods, and practices addressing society’s greatest challenges, and one of its kind.
May 12th 2022
I am presently visiting Professor Xin Li at the United States for a summer research visit. I will spend the entire summer working with him, and then finally I will attend the seventh International Conference on Continuous Optimization (ICCOPT)---one of the flagship venues for Optimization held in every three years. At ICCOPT, Prof. Elhoucine Bergou and I are organizing a session titled, "Stochastic optimization methods in Machine Learning" (covering topics from distributed computing, stochastic and deterministic optimization, algorithmic aspects for learning, approximation theory, and signal processing, etc.) at the Nonlinear Optimization Cluster. Additionally, I will give an invited talk in a session organized by Prof. Youssef Diouane, Prof. Serge Gratton, and Prof. Elhoucine Bergou.
September 29th 2021
Today is my last official working day at KAUST (King Abdullah University of Science and Technology), and the script can not be written any better. Special thanks to Panos Kalnis, Peter Richtárik, and my PhD adviser Xin Li for their immense support in this journey. Aritra out!
September 28th 2021
We got two papers accepted in Thirty-fifth Conference on Neural Information Processing Systems (NeuRIPS 2021)---Ranked 12 by Google Scholar among all conferences and journals across all fields.
- The first paper, "Rethinking gradient sparsification as total error minimization," managed to get in the top 3% out of 9122 submissions as a spotlight.
- The second paper, "DeepReduce: A Sparse-tensor Communication Framework for Federated Deep Learning," has been accepted as a poster.
July 19th 2021
I received the Best Reviewer Award (Top 10%) from the International Conference on Machine Learning 2021 (ICML 2021).
July 3rd 2021
I will start a new position as an Assistant Professor in the Department of Mathematics and Computer Science at the University of Southern Denmark (Syddansk Universitet, SDU) beginning October 2021.
May 15th 2021
I am organizing a special session at the INFORMS annual meeting, to be held in Anaheim, California, USA, October 24-27th 2021, jointly with El Houcine Bergou. The title of the session is "Stochastic Optimization Methods in Machine Learning". We have a wonderful list of speakers.
March 19th 2021
I received the "Outstanding Review Award" from International Conference on Learning Representation, 2021.
March 17th 2021
Our paper GRACE: A Compressed Communication Framework for Distributed Machine Learning got accepted in 41st IEEE International Conference on Distributed Computing Systems (ICDCS). For the full technical version of the paper please see Compressed Communication for Distributed Deep Learning: Survey and Quantitative Evaluation. The codebase is available here codebase-GRACE.
According to the ICDCS acceptance email "The selection process was very competitive this year" yielding an acceptance rate of 19.8 %.
February 4th 2021
I accepted an invitation to join the Editorial Board of Signal Processing Theory as Review Editor for Frontiers in Signal Processing.
February 8th 2021
New Paper out: DeepReduce: A Sparse-tensor Communication Framework for Distributed Deep Learning, joint work with Kelly Kostopoulou from Columbia University, Xin Li from University of Central Florida, Hang Xu and Panos Kalnis from KAUST, and Alexandros Ntoulas from National and Kapodistrian University of Athens.
Abstract: Sparse tensors appear frequently in distributed deep learning, either as a direct artifact of the deep neural network's gradients, or as a result of an explicit sparsification process. Existing communication primitives are agnostic to the peculiarities of deep learning; consequently, they impose unnecessary communication overhead. This paper introduces DeepReduce, a versatile framework for the compressed communication of sparse tensors, tailored for distributed deep learning. DeepReduce decomposes sparse tensors in two sets, values and indices, and allows both independent and combined compression of these sets. We support a variety of common compressors, such as Deflate for values, or run-length encoding for indices. We also propose two novel compression schemes that achieve superior results: curve fitting-based for values and bloom filter-based for indices. DeepReduce is orthogonal to existing gradient sparsifiers and can be applied in conjunction with them, transparently to the end-user, to significantly lower the communication overhead. As proof of concept, we implement our approach on Tensorflow and PyTorch. Our experiments with large real models demonstrate that DeepReduce transmits fewer data and imposes lower computational overhead than existing methods, without affecting the training accuracy.
January 21st 2021
New Paper out: A Fast and Adaptive SVD-free Algorithm for General Weighted Low-rank Recovery, joint work with Jingwei Liang of University of Cambridge, and Xin Li of University of Central Florida.
Abstract: This paper is devoted to proposing a general weighted low-rank recovery model and designs a fast SVD-free computational scheme to solve it. First, our generic weighted low-rank recovery model unifies several existing approaches in the literature. Moreover, our model readily extends to the non-convex setting. Algorithm-wise, most first-order proximal algorithms in the literature for low-rank recoveries, require computing singular value decomposition (SVD). As SVD does not scale properly with the dimension of the matrices, these algorithms becomes slower when the problem size becomes larger. By incorporating the variational formulation of the nuclear norm into the sub-problem of proximal gradient descent, we avoid computing SVD which results in significant speed-up. Moreover, our algorithm preserves the rank identification property of nuclear norm [33] which further allows us to design a rank continuation scheme that asymptotically achieves the minimal iteration complexity. Numerical experiments on both toy example and real-world problems including structure from motion (SfM) and photometric stereo, background estimation and matrix completion, demonstrate the superiority of our proposed algorithm.
January 12th 2021
I have accepted an invitation to serve as a Reviewer of International Conference on Computer Vision 2021 (ICCV 2021).
December 19th 2020
I have accepted an invitation to serve as a Reviewer of International Conference on Machine Learning 2021 (ICML 2021).
October 12th 2020
The public GITHUB repository called Distributed ML for our paper Huffman Coding Based Encoding Techniques for Fast Distributed Deep Learning is available. This repository provides essential encoding techniques used in our work. Any questions/comments are welcome.
November 9th 2020
I have accepted an invitation to serve as a Session Chair of the Machine Learning Session at the Twenty-Seventh International Conference on Neural Information Processing (ICONIP 2020).
October 12th 2020
Our paper Huffman Coding Based Encoding Techniques for Fast Distributed Deep Learning, joint work with Rishikesh R. Gajjala, Shashwat Banchhor of IIT Delhi and Ahmed Sayed, Marco Canini, and Panos Kalnis of KAUST got accepted for publication in ACM 16th International Conference on emerging Networking EXperiments and Technologies (CoNEXT), DistributeML workshop.
September 2020
I have accepted invitations to serve as a reviewer for the International Conference on Learning Representations (ICLR 2021), IEEE Computer Vision and Pattern Recognition 2021 (CVPR 2021), and as a Program Committee (PC) member of Thirty-Fifth AAAI Conference on Artificial Intelligence (AAAI-21).
July 7th 2020
Our paper Best Pair Formulation & Accelerated Scheme for Non-convex Principal Component Pursuit, joint work with Jingwei Liang of University of Cambridge, Filip Hanzley of KAUST, and Peter Richtárik of KAUST got accepted for publication as regular paper in the IEEE Transactions on Signal Processing (2018-'19 Impact Factor 6.083).
June 7th 2020
I have accepted an invitation to serve as a Program Committee (PC) member for the IEEE CVF Winter Conference on Applications of Computer Vision (WACV 2021).
May 8th 2020
I have accepted an invitation to serve as a Program Committee (PC) member for the Twenty-Seventh International Conference on Neural Information Processing (ICONIP 2020).
May 7th 2020
We have made our codebase-GRACE(GRAdient ComprEssion for distributed deep learning) public for the paper Compressed Communication for Distributed Deep Learning: Survey and Quantitative Evaluation. Any feedback and/or ideas for future collaboration around this topic is welcome!
April 13th 2020
New Paper out: Compressed Communication for Distributed Deep Learning: Survey and Quantitative Evaluation joint work with Hang Xu, Chen-Yu Ho, Ahmed Mohamed Abdelmoniem Sayed, El Houcine Bergou, Konstantinos Karatsenidis, Marco Canini, and Panos Kalnis of KAUST.
Abstract: Powerful computer clusters are used nowadays to train complex deep neural networks (DNN) on large datasets. Distributed training workloads increasingly become communication bound. For this reason, many lossy compression techniques have been proposed to reduce the volume of transferred data. Unfortunately, it is difficult to argue about the behavior of compression methods, because existing work relies on inconsistent evaluation testbeds and largely ignores the performance impact of practical system configurations. In this paper, we present a comprehensive survey of the most influential compressed communication methods for DNN training, together with an intuitive classification (i.e., quantization, sparsification, hybrid and low-rank). We also propose a unified framework and API that allows for consistent and easy implementation of compressed communication on popular machine learning toolkits. We instantiate our API on TensorFlow and PyTorch, and implement 16 such methods. Finally, we present a thorough quantitative evaluation with a variety of DNNs (convolutional and recurrent), datasets and system configurations. We show that the DNN architecture affects the relative performance among methods. Interestingly, depending on the underlying communication library and computational cost of compression/decompression, we demonstrate that some methods may be impractical.
April 7th 2020
New Paper out: On the Convergence Analysis of Asynchronous SGD for Solving Consistent Linear Systems joint work with Atal Narayan Sahu of KAUST, Aashutosh Tiwari of IIT, Kanpur, and Peter Richtárik of KAUST and MIPT.
Abstract: In the realm of big data and machine learning, data-parallel, distributed stochastic algorithms have drawn significant attention in the present days. While the synchronous versions of these algorithms are well understood in terms of their convergence, the convergence analyses of their asynchronous counterparts are not widely studied. In this paper, we propose and analyze a distributed, asynchronous parallel SGD in light of solving an arbitrary consistent linear system by reformulating the system into a stochastic optimization problem as studied by Richtárik and Takáč in [35]. We compare the convergence rates of our asynchronous SGD algorithm with the synchronous parallel algorithm proposed by Richtárik and Takáč in [35] under different choices of the hyperparameters---the stepsize, the damping factor, the number of processors, and the delay factor. We show that our asynchronous parallel SGD algorithm also enjoys a global linear convergence rate, similar to the basic method and the synchronous parallel method in [35] for solving any arbitrary consistent linear system via stochastic reformulation. We also show that our asynchronous parallel SGD improves upon the basic method with a better convergence rate when the number of processors is larger than four. We further show that this asynchronous approach performs asymptotically better than its synchronous counterpart for certain linear systems. Moreover, for certain linear systems, we compute the minimum number of processors required for which our asynchronous parallel SGD is better, and find that this number can be as low as two for some ill-conditioned problems.
March 23rd 2020
I have accepted an invitation to serve as a reviewer for the Thirty-fourth Conference on Neural Information Processing Systems 2020.
March 20th 2020
The website for 29th International Joint Conference on Artificial Intelligence and the 17th Pacific Rim International Conference on Artificial Intelligence, IJCAI-PRICAI 2020 tutorial proposal Compressed Communication for Large-scale Distributed Deep Learning is up and running.
March 6th 2020
Our tutorial proposal Compressed Communication for Large-scale Distributed Deep Learning jointly with El Houcine Bergou and Prof. Panos Kalnis got accepted in the 29th International Joint Conference on Artificial Intelligence and the 17th Pacific Rim International Conference on Artificial Intelligence, IJCAI-PRICAI 2020. The conference will be held in July 11-17th, 2020 in Yokohama, Japan. For the final schedule please visit IJCAI-PRICAI 2020. Stay tuned with the tutorial webpage and the online material. For any queries and details regarding the tutorial please contact me or Dr Bergou.
February 5th-13th 2020
I will be attending the Thirty-Fourth AAAI Conference on Artificial Intelligence (AAAI-20) from February 5th till February 13th, 2020. The conference will be held at the Hilton New York Midtown. I will present my joint work accepted and designated for spotlight and poster presentation at AAAI 2020 on Thursday, February 11th at the 55th Session from 11:15 am-12:30 pm, Paper ID 2100, titled: On the Discrepancy between the Theoretical Analysis and Practical Implementations of Compressed Communication for Distributed Deep Learning We will present in the poster session on the evening of the same day (11th February) as our poster spotlight talk. Feel free to get in touch to discuss about any future collaboration or to just grab a coffee.
December 2019
I am organizing a special session at the SIAM conference on Optimization 2020 (SIOPT 2020) to be held at Hong Kong in May 2020. SIOPT is a flagship conference series on mathematical optimization, covering a wide range of topics in the field. For more details about the conference visit this webpage.
November 20th 2019
I have accepted an invitation to serve as a reviewer for the International Conference on Machine Learning 2020 (ICML 2020).
November 19th 2019
New Paper out: On the Discrepancy between the Theoretical Analysis and Practical Implementations of Compressed Communication for Distributed Deep Learning joint work with El Houcine Bergou, Ahmed Mohamed Abdelmoniem Sayed, Chen-Yu Ho, Atal Narayan Sahu , Marco Canini, and Panos Kalnis.
Venue: To appear in Thirty-Fourth AAAI Conference on Artificial Intelligence (AAAI-20).
Abstract: Compressed communication, in the form of sparsification or quantization of stochastic gradients, is employed to reduce communication costs in distributed data-parallel training of deep neural networks. However, there exists a discrepancy between theory and practice: while theoretical analysis of most existing compression methods assumes compression is applied to the gradients of the entire model, many practical implementations operate individually on the gradients of each layer of the model.
In this paper, we prove that layer-wise compression is, in theory, better, because the convergence rate is upper bounded by that of entire-model compression for a wide range of biased and unbiased compression methods. However, despite the theoretical bound, our experimental study of six well-known methods shows that convergence, in practice, may or may not be better, depending on the actual trained model and compression ratio. Our findings suggest that it would be advantageous for deep learning frameworks to include support for both layer-wise and entire-model compression.
November 11th 2019
Our paper On the Discrepancy between the Theoretical Analysis and Practical Implementations of Compressed Communication for Distributed Deep Learning, joint work with El Houcine Bergou, Ahmed M. Ab, Chen-Yu Ho, Atal Narayan Sahu, Marco Canini, and Panos Kalnis, got accepted in Thirty-Fourth AAAI Conference on Artificial Intelligence (AAAI-20).
According to the AAAI acceptance email there was a record number of over 8,800 submissions this year. Of those, only 1,591 papers got accepted, yielding an acceptance rate of 20.6%.
September 18th 2019
I have accepted an invitation to serve as a reviewer for the IEEE Computer Vision and Pattern Recognition 2020 (CVPR 2020).
August 16th 2019
I have accepted an invitation to serve as a Program Committee (PC) member for the Thirty-Fourth AAAI Conference on Artificial Intelligence (AAAI-20).
August 2nd-9th 2019
I am now in Berlin, Germany, organizing three special sessions at the Sixth International Conference on Continuous Optimization 2019, jointly with El Houcine Bergou. Title of the sessions are: "New Trends in Optimization Methods and Machine Learning I, II, and III".
June 6th 2019
I will be attending the Thirty-sixth International Conference on Machine Learning (ICML) from June 10th till 15th, 2019 to be held at the Long Beach Convention Center, Long Beach, Los Angeles, California. Our group will present several papers. Feel free to get in touch to discuss any future collaboration or to just grab a coffee.
May 26th 2019
New Paper out: Direct Nonlinear Acceleration, joint work with El Houcine Bergou of KAUST and INRA, Marco Canini of KAUST, Yunming Xiao of Tsinghua University, and Peter Richtárik of KAUST and MIPT.
Abstract: Optimization acceleration techniques such as momentum play a key role in state-of-the-art machine learning algorithms. Recently, generic vector sequence extrapolation techniques, such as regularized nonlinear acceleration (RNA) of Scieur et al., were proposed and shown to accelerate fixed point iterations. In contrast to RNA which computes extrapolation coefficients by (approximately) setting the gradient of the objective function to zero at the extrapolated point, we propose a more direct approach, which we call direct nonlinear acceleration (DNA). In DNA, we aim to minimize (an approximation of) the function value at the extrapolated point instead. We adopt a regularized approach with regularizers designed to prevent the model from entering a region in which the functional approximation is less precise. While the computational cost of DNA is comparable to that of RNA, our direct approach significantly outperforms RNA on both synthetic and real-world datasets. While the focus of this paper is on convex problems, we obtain very encouraging results in accelerating the training of neural networks.
May 26th 2019
New Paper out: Best Pair Formulation & Accelerated Scheme for Non-convex Principal Component Pursuit, joint work with Jingwei Liang of University of Cambridge, Filip Hanzley of KAUST, and Peter Richtárik of KAUST and MIPT.
Abstract: The best pair problem aims to find a pair of points that minimize the distance between two disjoint sets. In this paper, we formulate the classical robust principal component analysis (RPCA) as the best pair; which was not considered before. We design an accelerated proximal gradient scheme to solve it, for which we show global convergence, as well as the local linear rate. Our extensive numerical experiments on both real and synthetic data suggest that the algorithm outperforms relevant baseline algorithms in the literature.
March 12th 2018
I have accepted an invitation to serve as a reviewer for the Thirty-third Conference on Neural Information Processing Systems 2019.
March 8th 2019
I am organizing two special sessions at the Sixth International Conference on Continuous Optimization 2019, to be held in Berlin, Germany, August 3-8th 2019, jointly with El Houcine Bergou. We will start sending out the invitations soon. Title of the session is "New Trends in Optimization Methods and Machine Learning I and II".
ICCOPT is a flagship conference series of the Mathematical Optimization Society (MOS) on continuous optimization, covering a wide range of topics in the field. Our sessions are part of the “Non-Linear Optimization” cluster.
January 24th 2019
I will be attending the Thirty-Third AAAI Conference on Artificial Intelligence from January 27th till February 1st, 2019. The conference will be held at the Hilton Hawaiian Village, Honolulu, Hawaii. I will present my joint work with Filip Hanzley and Peter Richtárik accepted and designated for oral presentation at AAAI 2019 on January 30th - Paper ID 5833 titled: A Nonconvex Projection Method for Robust PCA. We will present in the poster session as well. Feel free to get in touch to discuss about any future collaboration or to just grab a coffee. Let's get the party started as ICML deadline is over.
January 13th 2019
I am back to KAUST and working on ICML deadline. I am unable to join American Mathematical Society's Joint Mathematics Meeting 2019 to be held in Baltimore, USA, January 16-19, 2019 due to extremely busy travel and research schedule in this month (two Hawaii trips and ICML deadline!!). But our session "Optimal Methods in Applicable Analysis: Variational Inequalities, Low Rank Matrix Approximations, Systems Engineering, Cyber Security" (SS-81) is going to ROCK! I thank all the special session invitees and apologize to my old friends for not catching up with you in JMM this year.
January 5th 2019
I will be attending the WACV 2019 – IEEE Winter Conference on the Applications of Computer Vision from January 7th till January 10th, 2019. The conference will be held at the Hilton Waikoloa Village, Hawaii. I will present my joint work with Peter Richtárik accepted at WACV- Online and Batch Supervised Background Estimation via L1 Regression (paper ID 84) as a 5 minutes spotlight talk in the oral Session 2-A titled "Visions Systems and Applications" (on Tuesday, 8th January 2019, 3:20-5:00pm) followed by a two hour poster session. I am on my way to Hilton Waikoloa Village. Feel free to get in touch to talk about my present work and for future collaboration.
December 14th 2018
Our paper A Nonconvex Projection Method for Robust PCA is designated for oral presentation at the Thirty-Third AAAI Conference on Artificial Intelligence (AAAI-19). The email says : "papers were chosen for the conference according to a uniform standard of technical contribution. Selection of presentation format for accepted papers was based primarily on an assessment of breadth of interest, and the construction of balanced and topically coherent sessions."
November 14th 2018
Finally, I am back to KAUST after attending INFORMS and spending my vacation. I presented a poster based on our recent work A Nonconvex Projection Method for Robust PCA at the Statistics and Data Science workshop at KAUST.
November 4th 2018
Our paper Online and Batch Supervised Background Estimation via L1 Regression got accepted in WACV 2019 – IEEE Winter Conference on the Applications of Computer Vision. It was a joint work with Peter Richtárik of KAUST, MIPT, and University of Edinburgh. The conference will take place in Waikoloa Village, Hawaii, USA, January 7-11, 2019.
November 1st 2018
I am attending the 2018 INFORMS Annual Meeting. I am also chairing two sessions, jointly with Dr El Houcine Bergou. Title of our sessions are: "Stochastic Optimization Methods and Approximation Theory in Machine Learning I and II".
October 31st 2018
Our paper A Nonconvex Projection Method for Robust PCA, joint work with Filip Hanzley of KAUST and Peter Richtárik of KAUST, MIPT, University of Edinburgh got accepted in Thirty-Third AAAI Conference on Artificial Intelligence (AAAI-19).
According to the AAAI acceptance email there was a record number of over 7,700 submissions this year. Of those, 7,095 were reviewed, and only 1,150 papers got accepted, yielding an acceptance rate of 16.2%. The email quotes "There was especially stiff competition this year because of the number of submissions, and you should be proud of your success. " The conference will take place in Honolulu, Hawaii, USA, January 27-February 1, 2019.
October 19th 2018
I have accepted an invitation to serve as a reviewer for The 36th International Conference on Machine Learning (ICML 2019) which is to be held in Long Beach, California, June 10-15, 2019. I am looking forward to learn many exciting things by reviewing some cutting edge work. Paper submission deadline: January 23, 2019
June 4th-9th 2018
I am attending SIAM Conference on Imaging Science 2018 at Bologna, Italy. I will give a talk at the minisymposia Computational Methods for Large-Scale Machine Learning in Imaging. It is an amazing place and the conference is great so far. I am excited to see some of my old friends and collaborators.
May 19th 2018
New Paper out: A Nonconvex Projection Method for Robust PCA, joint work with Filip Hanzley of KAUST and Peter Richtárik of KAUST, MIPT, University of Edinburgh.
Abstract: Robust principal component analysis (RPCA) is a well-studied problem with the goal of decomposing a matrix into the sum of low-rank and sparse components. In this paper, we propose a nonconvex feasibility reformulation of RPCA problem and apply an alternating projection method to solve it. To the best of our knowledge, we are the first to propose a method that solves RPCA problem without considering any objective function, convex relaxation, or surrogate convex constraints. We demonstrate through extensive numerical experiments on a variety of applications, including shadow removal, background estimation, face detection, and galaxy evolution, that our approach matches and often significantly outperforms current state-of-the-art in various ways.
May 12th 2018
We are overwhelmed by the extraordinary response from the invitees. Now we have two special sessions at 2018 INFORMS annual meeting: "Stochastic Optimization Methods and Approximation Theory in Machine Learning-I and II". Dr Bergou and I are thankful to those who participated.
May 9th 2018
I am organizing a special session jointly with Prof. Ram N. Mohapatra at the American Mathematical Society's Joint Mathematics Meeting 2019 to be held in Baltimore, USA, January 16-19, 2019. The title of the special session is "Optimal Methods in Applicable Analysis: Variational Inequalities, Low Rank Matrix Approximations, Systems Engineering, Cyber Security" (SS 81). We are grateful to the AMS for granting us a total 7 hours for our session. JMM is world's largest mathematics meeting with 6400+ participants in last year's annual meeting. It is going to be very exciting. We are going to send out the invitations soon. I also thank Dr Mohapatra for working on the proposal with me.
May 4th 2018
Our special session "Stochastic Optimization Methods and Approximation Theory in Machine Learning" to be held at INFORMS 2018 annual meeting has five extraordinary speakers. Dr Bergou and I are thankful to those who participated. The list of the speakers is coming soon! I am looking for two more speakers which can result in to another session. If anyone is interested in participating please contact me.
May 3rd 2018
I graduated with the Udacity Deep Learning Nanodegree! I learned some great stuff and did some exciting projects. Looking forward to doing some hands on work in Deep Learning.
April 18th 2018
I am organizing a special session at the INFORMS annual meeting, to be held in Phoenix, Arizona, November 4-7th 2018, jointly with El Houcine Bergou. We will start sending out the invitations soon. Title of the session is "Stochastic Optimization Methods and Approximation Theory in Machine Learning".
April 15th 2018
New Paper out: Weighted Low-Rank Approximation for Background Modeling, joint work with Xin Li of UCF and Peter Richtárik of KAUST.
Abstract: We primarily study a special a weighted low-rank approximation of matrices and then apply it to solve the background modeling problem. We propose two algorithms for this purpose: one operates in the batch mode on the entire data and the other one operates in the batch-incremental mode on the data and naturally captures more background variations and computationally more effective. Moreover, we propose a robust technique that learns the background frame indices from the data and does not require any training frames. We demonstrate through extensive experiments that by inserting a simple weight in the Frobenius norm, it can be made robust to the outliers similar to the L1 norm. Our methods match or outperform several state-of-the-art online and batch background modeling methods in virtually all quantitative and qualitative measures.
March 12th 2018
I went to attend the workshop Integrating Machine Learning and Predictive Simulation: From Uncertainty Quantification to Digital Twins held in Institute for Mathematics and its Applications (IMA), 4-8th March, 2018. Thanks to the organizers and generous funding of IMA for providing me with a comprehensive travel grant. I had a great time at Minneapolis, Minnesota.
March 9th 2018
I gave an alumni talk at the Mathematics department, University of Central Florida.
May 2017
I was fortunate to be featured as one of the successful graduate students on the University of Central Florida's Mathematics department's doctoral program's brochure. It has been a great honor. I was also fortunate to be featured on the University of Central Florida's Graduate Catalog.
April 2017
I was selected as a winner of Lee H. Armstrong award for excellence in Graduate teaching, 2017.
16th December 2016
I graduated with a PhD in Mathematics. Thanks to my advisors: Prof. Xin Li and Prof. Qiyu Sun. I also thank my other committee members: Prof. Mubarak Shah, Prof. Ram N Mohapatra, and Prof. Zuhair Nashed. I also thank my dearest friends and family members. Without their support and love nothing would have been possible.
November 2016
I won the Outstanding Dissertation Award 2016 from the Mathematics Department, University of Central Florida.
EDUCATION
Ph.D. in Mathematics, University of Central Florida, USA, December 2016.
M.S. in Mathematics, University of Central Florida, USA, December 2011.
M.S. in Mathematics and Computing, Indian Institute of Technology, Dhanbad, June 2008.
B.S. in Mathematics (Hons.), Minor in Physics and Statistics, Presidency College, Calcutta University, June 2006.
M.S. in Mathematics, University of Central Florida, USA, December 2011.
M.S. in Mathematics and Computing, Indian Institute of Technology, Dhanbad, June 2008.
B.S. in Mathematics (Hons.), Minor in Physics and Statistics, Presidency College, Calcutta University, June 2006.
