Machine learning markets

Abstract: In the context of building a machine learning framework that scales, the current modus operandi is a monolithic, centralised model building approach. These large scale models have different components, which have to be designed and specified in order to fit in with the model as a whole. The result is a machine learning process that needs a grand designer. It is analogous to a planned economy. There is an alternative. Instead of a centralised planner being in charge of each and every component in the model, we can design incentive mechanisms for independent component designers to build components that contribute to the overall model design. Once those incentive mechanisms are in place, the overall planner need no longer have control over each individual component. This is analogous to a market economy. The result is a transactional machine learning. The problem is transformed to one of setting up good incentive mechanisms that enable the large scale machine learning models to build themselves. One such framework uses machine learning markets. It turns out there is a direct relationship between machine learning markets with agents who minimize risk measures and distributed optimization methods via the generalized Fenchel duality. Different primal-dual optimization methods relate to different market dynamics. As part of work in progress we consider a primal-dual optimization that provides potential improvements to the Alternating Direction Method of Multipliers.