Whats new in Spark this week #2
I've been somewhat distracted and well there has been a several week lapse between updates for what I intended to be weekly (and that is only after the first one). That being said I'm going to take another crack at this at hopefully get a streak of longer than one.
Lets look at what has happened in the past week on spark:
Lets look at what has happened in the past week on spark:
- karenfeng made a number of UI improvements (https://github.com/mesos/spark/commit/b2aaa1199e7ecd8e1b2a9ddd8356b6393edafe6b , https://github.com/mesos/spark/commit/0d78b6d9cd11fc12c546f25fa857ba8b285c062d , https://github.com/mesos/spark/commit/e3d3e6f0ab34f0fe083ef9feb31b9e9fd257519f , https://github.com/mesos/spark/commit/39557112501901da7f9b4be6159a5a0be5511b42 ...)
- mateiz added a new option, spark.kryo.referenceTracking, which allows users to disable kyro's reference tracking which can be done if you know your objects do not contain any loops. https://github.com/mesos/spark/commit/4698a0d6886905ef21cbd52e108d0dcab3df12df
- pwendell has added an Environment tab to the UI to help track down issues with different JARs on the classpath, system properties not being set, and other enviroment related problems https://github.com/mesos/spark/pull/699
- Information about the RDD blocks locations was added to the status page https://github.com/mesos/spark/pull/697/commits by pwendell
- takeOrdered was added by Reinvigorate allowing one to get back the first k elements as defined by the provided ordering in https://github.com/mesos/spark/pull/692 & https://github.com/mesos/spark/commit/ee4ce2fc51112387f28d7b422969ca2e9736e95f
- The README was updated to reflect that the current version now requires Scala 2.9.3
Comments