MR3

A new execution engine for Hadoop and Kubernetes

Hive on MR3

Hive running on top of MR3

Hive on Kubernetes

Hive powered by MR3 on Kubernetes

MR3

MR3 is a new execution engine for Hadoop and Kubernetes. It is a new execution engine with simple design, excellent performance, and native support for both Yarn and Kubernetes. MR3 is ready for production use as it supports all major features such as Kerberos-based security, authentication and authorization, fault-tolerance, and recovery. MR3 is implemented in Scala.

Hive on MR3

Hive, the de facto standard for SQL queries in Hadoop, currently supports three execution engines for its backend -- MapReduce, Tez, and Spark. Now Hive can run on top of MR3 as well. Hive on MR3 runs much faster than Hive on Tez by virtue of the simple architectural design of MR3. In particular, it yields a much higher throughput for concurrent queries by making a better utilization of computing resources. Hive 2 and 3 also support an execution mode called LLAP (Low Latency Analytical Processing) designed for interactive queries. In comparison with Hive with LLAP, Hive on MR3 allows elastic allocation of cluster resources, provides better support for concurrency, and fully implements impersonation.

Hive on Kubernetes

With its native support for Kubernetes, MR3 can power Hive on Kubernetes clusters. As it creates and destroys containers as necessary, MR3 enables Hive to take full advantage of the dynamic resource allocation of Kubernetes. By reading data from remote sources while running containers on Kubernetes, Hive on MR3 can achieve the separation of compute and storage.


Contact Us