View article

In many ways, the objective of the IBM Parallel Machine Learning Toolbox (PML) is similar to that of Google’s MapReduce programming model (Dean and Ghemawat, 2004) and the open source Hadoop system, 1 which is to provide Application Programming Interfaces (APIs) that enable programmers who have no prior experience in parallel and distributed systems to nevertheless implement parallel algorithms with relative ease. Like MapReduce and Hadoop, PML supports associative-commutative computations as its primary parallelization mechanism. Unlike MapReduce and Hadoop, PML fundamentally assumes that learning algorithms can be iterative in nature, requiring multiple passes over data. It also extends the associative-commutative computational model in various aspects, the most important of which are:

1. The ability to maintain the state of each worker node between iterations, making it possible, for example, to partition and distribute data structures across workers 2. Efficient distribution of data, including the ability for each worker to read a subset of the data, to sample the data, or to scan the entire dataset 3. Access to both sparse and dense datasets 4. Parallel merge operations using tree structures for efficient collection of worker results on very large clusters