Kudu – the end of MapReduce?

The latest in a long line of struggling Data Warehouse companies announced their results  on 5th November. Teradata had flat revenues and flat to slightly declining revenues predicted for the year. Look at the Dell/EMC idea, HP”s continuing machinations and flat revenues, IBM’s problems (including the strange decision to buy …. The Weather Channel ), Oracle’s well documented problems  and it’s clear the seismic shift in Enterprise software that started with Virtualization, then Cloud and continues with Hadoop by way of open source continues to grind down the Proprietary Enterprise software vendors.

As Mike Olson put it over 2 years ago – the Proprietary vendors now have no hope of competing. They don’t have the resources, the knowledge or the skills to compete with globalized software development teams running Apache projects or similar.

So when a client asks us – what does this all mean? Why is everything moving so fast? Why can’t we get back on the old familiar proprietary software merry go round? All this uncertainty makes customers very nervous about what to do, and in some cases they therefore….. do nothing.

When a client asks this question we tell them three things :-

1. Look at what is happening in the critical Apache projects. Take Hadoop, or Spark, or Kudu or many others. Look at the trend in committers to see which projects have a solid bench of committers and is growing.
    
2. Secondly – look at where these committers are doing their day jobs. Here is a good example of Spark. Scroll down the list and you will see it’s solidly dominated by Databricks, with a good mix of UC Berkeley (where Spark was developed), Intel (watching their investment in Cloudera) and, of course Cloudera.
    
3. Thirdly – if you are intent on looking at projects still in incubation – be careful. Hadoop Development tools  seemed like a good idea but failed to get any sort of traction with the community.
    

The last piece of advice we give clients is always look at what the market leaders are donating to Apache. Last week Cloudera announced they want to donate Impala and Kudu to Apache. Impala has been widely adopted and so Cloudera want the community to take over some of the development work. Kudu is a big complex project so, now that it’s architecture has been broadly set, Cloudera want the bigger, more agile, more skilled worldwide community to shoulder some of the development load.

For many clients, used to easy decision making provided by Gartner and others, Open Source adoption in the enterprise and in the data center in particular is changing everything. 5 years ago and certainly 10 years ago clients would insist on Roadmaps under NDA, long term support with access to source code if the vendor went out of business, long complex discussions about pricing and licensing and entry into elite customer user groups. Not one of those types of discussion happens any more in Open Source and Big Data. No roadmaps (it’s up to the community to direct the roadmap so join in!), source code is already freely available, licensing is easy – it’s free!,

We believe clients need to adapt not just their technology options but also their vendor outlook to get the most out of the Open source world.

What do you think – are customers able to adapt to the changing market and technology landscape? Or they stuck in a time warp – working in a style and with a market view that no longer exists?

And take a look at the Platinum sponsors of ASF – they are sponsoring for a reason of course.