Mathematical optimization is the heart of any machine learning algorithm. Optimization problems consists of derivative-less-optimization, game theoretic models optimization, uncertainty based optimization, complementarity optimization and mixed integer optimization etc. However, the growth of data is exponential in these days and machine learning algorithmic complexity is also rapidly increasing. These factors are giving new challenges to optimization techniques in machine learning. A significant progress in the optimization techniques has been mode in this decade. In this chapter we can understand the optimization problems classification in machine learning along with different basic optimization methods (i.e. gradient descent, stochastic gradient, higher order optimization etc.). Thus, different machine learning optimization problems are explored and also explained different optimization methods along with machine learning optimization challenges.