Release Notes

Release 0.9: 2019-07-25

  • Each DAG uses its own ClassLoader.

Release 0.8: 2019-06-22

  • A new DAGAppMaster properly recovers DAGs that have not been completed in the previous DAGAppMaster.
  • Fault tolerance after fetch failures works much faster.
  • On Kubernetes, the shutdown handler of DAGAppMaster deletes all running Pods.
  • On both Yarn and Kubernetes, MR3Client automatically connects to a new DAGAppMaster after an initial DAGAppMaster is killed.

Release 0.7: 2019-04-26

  • Resolve deadlock when Tasks fail or ContainerWorkers are killed
  • Support fault tolerance after fetch failures
  • Support node blacklisting

Release 0.6: 2019-03-21

  • DAGAppMaster can run in its own Pod on Kubernetes.
  • Support elastic execution of RuntimeTasks in ContainerWorkers
  • MR3-UI requires only Timeline Server.

Release 0.5: 2019-02-18

  • Support Kubernetes
  • Support the use of a built-in shuffle handler

Release 0.4: 2018-10-29

  • Support auto parallelism for reducers with ONE_TO_ONE edges
  • Auto parallelism can use input statistics when reassigning partitions to reducers.
  • Support ByteBuffer sharing among RuntimeTasks

Release 0.3: 2018-08-15

  • Extend the runtime to support Hive 3

Release 0.2: 2018-05-18

  • Support asynchronous logging (mr3.async.logging in mr3-site.xml)
  • Delete DAG-local directories after each DAG is finished

Release 0.1: 2018-03-31

  • Initial release