### Blogroll

### Archives

- January 2017 (1)
- November 2016 (1)
- July 2016 (2)
- June 2016 (3)
- May 2016 (1)
- June 2012 (3)
- May 2012 (1)
- May 2011 (1)
- March 2011 (1)
- January 2011 (5)
- December 2010 (2)
- November 2010 (2)

### Categories

- Readings (13)
- Research (22)
- Algorithm (9)
- Cryptogaphy (2)
- Game Theory (1)
- Learning Theory (6)
- Optimization (8)

- Story (2)

# Category Archives: Optimization

## SVD and Eigenvectors for Big Data

People generally believe that a PC can handle SVD for matrices only up to thousands by thousands. Textbooks also suggest that it is not wise to compute singular vectors one by one. In this post, I’ll refute both statements. In particular, … Continue reading

Posted in Algorithm, Optimization
Leave a comment

## Faster Than SGD 2: the Katyusha Acceleration

SGD is well-known for large-scale optimization. In my mind, there are so-far two fundamentally different improvements since the original introduction of SGD: (1) variance reduction, and (2) acceleration. The following picture predicts —in theory— the performance difference between the naive … Continue reading

Posted in Optimization
1 Comment

## Faster Than SGD 1: Variance Reduction

SGD is well-known for large-scale optimization. In my mind, there are two (and only two) fundamental improvements since the original introduction of SGD: (1) variance reduction, and (2) acceleration. In this guest post at Princeton’s OptiML group, I’d love to conduct … Continue reading

Posted in Algorithm, Learning Theory, Optimization
Leave a comment

## Faster Comp-Geometry via Optimization

If one wants to compute the minimum enclosing ball (MinEB) of a set of points, would you believe that the running time can be improved by a significant factor if we randomly rotate the space? While seemingly very counter-intuitive because a … Continue reading

Posted in Algorithm, Optimization
Leave a comment

## ICML posters available!

The third one is mentioned in this blog post. The other two are coming! PS: the video of the three talks are on YouTube now.

Posted in Optimization
Leave a comment

## Coordinate Descent in One Line, or Three if Accelerated

When minimizing a convex function using first-order methods, if full gradients are too costly to compute at each iteration, there are two alternatives that can reduce this per-iteration cost. One is to use a (random) coordinate gradient , and the other is … Continue reading

Posted in Optimization
Leave a comment

## How to solve classification and regression fast, faster, and fastest

I am often asked what is the best algorithm to solve SVM, to solve Lasso Regression, to solve Logistic Regression, etc. At the same time, a growing number of first-order methods have been recently proposed, making even experts hard to … Continue reading

Posted in Optimization
Leave a comment