Friday, January 23, 2015

M$ Acquires Revolution Analytics

Heard from multiple people the news from today. Revolution is based on an open source model of support for R users.

Friday, January 16, 2015

University of Cambridge donates improved graph coloring code to PowerGraph

Philip Leonard, a student from the University of Cambridge has just donated improved graph coloring implementations to PowerGraph. Here is the tech report describing his project.
As we are soon going to release GraphLab Create as our newer open source repo we hope to get additional contributions to there as well!

Tuesday, January 13, 2015

Is MLLib being deprecated?

I got this from my colleague Krishna Sridhar. It seems that a new ML library, spark.ml is being written on top of Spark with the goal of deprecating MLlib.

If all goes well, spark.ml will become the primary ML package at the time of the Spark 1.3 release. Initially, simple wrappers will be used to port algorithms to spark.ml, but eventually, code will be moved to spark.ml and spark.mllib will be deprecated.
I just got a note from Xiangrui Meng, who is heading this effort. It seems the above text was not clear. Here is a clarification of their new plan:

spark.ml contains high-level APIs for building ML pipelines. But it doesn't mean that spark.mllib is being deprecated, nor MLlib as a Spark component is being deprecated. First of all, the spark.ml pipeline API is in its alpha stage and we need to see more use cases from the community to stabilizes it. Secondly, the components in spark.ml are simple wrappers over spark.mllib implementations. Neither the APIs nor the implementations from spark.mllib are being deprecated. We expect users use spark.ml pipeline APIs to build their ML pipelines, but we will keep supporting and adding features to spark.mllib. For example, you can find many features in review at https://spark-prs.appspot.com/#mllib. So users should be comfortable with using spark.mllib features and expect more features coming. I will update the user guide to make the message clear. Thanks for bringing this up! 

Thursday, January 8, 2015

Tuesday, January 6, 2015

O'Reilly Strata Survey: Technology and Salaries

My colleague Alice Zheng sent me the following link to Strata salary and technology survey 2014.

Here are some interesting plots taken out of this survey.