MapR’s Spyglass keeps an eye on big data deployments

At Hadoop Summit in San Jose Tuesday Hadoop distribution vendor MapR Technologies took the wraps off a new initiative intended to maximize the productivity of users and administrators of big data deployments.

“This is putting some focus on the user and operator side,” says Dale Kim, senior director of Product Marketing at MapR. “It provides deep visibility and full control.”

MapR calls it the Spyglass Initiative — a comprehensive, open and extensible approach to centralized monitoring of big data deployments.

[ Related: MapR shows off enterprise-grade Spark distribution ]

Kim says the first phase of the Spyglass Initiative will be included in the upcoming release of the MapR Converged Data Platform. The new features include the following:

• Deep search across cluster-wide metrics and logs. This new functionality, built on Elasticsearch, integrates tools for aggregating and storing metrics and log data from MapR, providing deep visibility into a big data cluster to help plan next steps.
• Shareable, customizable, mobile-ready, multi-tenant dashboards. New dashboards provide a complete view of cluster operations in user-defined formats. By leveraging the newly launched Exchange within the MapR Converge Community, customers can share custom dashboards with peers for a variety of visualization tools, including Grafana and Kibana. “These dashboards allow you to mix different types of metrics together into one view,” Kim says. “It lends itself well to predicting what’s going on with your cluster, what resource allocations you’ll have to make and what types of load you’ll have to accommodate.”
• Extensible APIs for third-party tool integration. Via open APIs, organizations can also leverage other options for visualizing their data. MapR enables monitoring nodes/infrastructure (including read/write throughput and database operations), cluster space utilization, YARN/MapReduce applications and service daemons.

MapR also moved to simplify planning and maintaining big data deployments with the MapR Ecosystem Pack (MEP), a program through which it decouples its platform releases from open source project releases and certifies Hadoop ecosystem components for interoperability.

[ Related: MapR delivers support for containers, security ]

“This is about making sure our customers have quick access to the latest projects,” Kim says. “you can have the latest version of Spark and Drill, but also the right versions of other projects to ensure interoperability. We do extensive testing so customers don’t need to figure out which versions of projects work with other projects.”

MapR has already been giving customers monthly updates to new project versions. Kim says MEP is the next evolution of that program, providing full certification on cross-project interoperability. With MEP, MapR will release certified packs of open source projects on a quarterly basis. It will also continue delivering updates on a monthly basis to address open source project bug fixes.

“When we have a major platform release, we’ll release a major MEP release along with it,” Kim says. “We’re giving [customers] this certified and blessed package of what projects work together. The end result is it will make it a lot easier for customers to decide what to upgrade and when to upgrade.”

The new MapR Platform will also include performance and JSON enhancements to MapR-DB. Advanced multi-master JSON replication provides mission-critical disaster recovery for JSON documents and a global view of enterprise-wide data in local deployments.