SoylentNews is people
SoylentNews
The Hadoop dream of unifying data and compute in a distributed manner has all but failed in a smoking heap of cost and complexity, according to technology experts and executives who spoke to Datanami.
"I can't find a happy Hadoop customer. It's sort of as simple as that," says Bob Muglia, CEO of Snowflake Computing, which develops and runs a cloud-based relational data warehouse offering. "It's very clear to me, technologically, that it's not the technology base the world will be built on going forward."
Thousands of organizations store huge amounts of data in Hadoop, and so Hadoop won't disappear overnight. After all, many companies still run mainframe applications that were originally developed half a century ago. But thanks to better mousetraps like S3 (for storage) and Spark (for processing), Hadoop will be relegated to niche and legacy statuses going forward, Muglia says.
"The number of customers who have actually successfully tamed Hadoop is probably less than 20 and it might be less than 10," Muglia says. "That's just nuts given how long that product, that technology has been in the market and how much general industry energy has gone into it."
Are any Soylentils using Hadoop, or deliberately using alternatives, for their Big Data needs?
(Score: -1, Troll) by Anonymous Coward on Tuesday March 28 2017, @11:22AM
Fork it.
(Score: 0) by Anonymous Coward on Tuesday March 28 2017, @01:06PM (1 child)
So he's officially a special snowflake? ;-)
(Score: 2) by DannyB on Tuesday March 28 2017, @04:00PM
Just an ordinary snowflake with typical specifications. It's not Special Snowflake Computing.
If you eat an entire cake without cutting it, you technically only had one piece.
(Score: 3, Informative) by Anonymous Coward on Tuesday March 28 2017, @01:25PM (2 children)
I have visibility into a lot of real-world installations of Hadoop, combined with (other technology, I won't tell you what, I'm not authorised to speak for my employers, blahblahblabh) and this is just not true.
A large number of serious organisations put a lot of data into Hadoop installations, analyse it in a number of complex ways, and come out with answers that they desired. That's a success.
Sure, Hadoop has limitations - most notably, it's an isolated bear of a thing to use - but it's quite good at what it does.
(Score: 4, Insightful) by Anonymous Coward on Tuesday March 28 2017, @03:16PM (1 child)
Well, from the summary, emphasis by me:
CEO claims that software competing with their own offering is bad. News at 11.
(Score: 0) by Anonymous Coward on Tuesday March 28 2017, @04:33PM
yep, sounds like fud to me.
(Score: 2) by hendrikboom on Tuesday March 28 2017, @02:24PM (1 child)
If you are serious about building a completely distributed system, try building it on Inferno [wikipedia.org]
(Score: 0) by Anonymous Coward on Wednesday March 29 2017, @06:51AM
(Score: 3, Interesting) by bootsy on Tuesday March 28 2017, @05:17PM
It feels like the documentation is very poor and that you need to give Hortonworks a lot of money for consultants to configure it.
In most corporate environments you want the whole thing plugged into LDAP and Kerberos and the setup is tricky and non intuitive.
I've been using Apache Storm recently and part of the problem with these technologies is it hard to decide which one to use.
Should I use Spark, Storm, Kafka, Mongo or HBase. There isn't a clear use case or best fit for what suits your problem the best.
It's difficult to hire for it as well when most Java developers just seem to use the Spring Framework for everything. The skills for these technologies are not widely known, especially outside of a certain class of problems.
Using a non relational database in favour of a 1970s style hierachical design is always going to give problems when you want to run ad-hoc queries. The traditional business analyst is not a data scientist and just isn't used to working with these tools.
How you handle an error or a failure is important. If Google miss a link or twitter miss a tweet then no-one dies and it isn't the end of the world. In a finance or billing application then this would cause a big problem.
I think it is still too early to make a call on how Hadoop will evolve but I see a lot of proof of concepts projects but not a lot of actual deployments and go-lives.