Will Hadoop Work for Medium-Sized Data?

Analytics | Co-creation | Published 09. Jun. 16

Share with Your Professional Network

ADASTRA specializes in Information Management offering comprehensive and targeted services that streamline business processes, upgrade technology infrastructure, and unlock data assets and investment values. In this case study, the challenge was to find out if Hadoop will also work on medium-sized data.


The Need for a Regular Calculation of Credits 

The case company is one of the biggest banks in Germany. It’s dissatisfied with the current C++ & SQL implementation performance and it needs to calculate the value of its credits on a regular basis. The calculation consists of statistical formulas and joins/aggregations/searches.

Data Transfer to Hadoop

The solution was to set up a Hadoop cluster at the case company’s premises and transfer the necessary 10 GB data from its database to the Hadoop cluster. Then, the specification given by the customer was re-implemented in Apache Spark/Python (statistical calculations) and Impala (joins/aggregations). Finally, the configuration and execution order were optimized to maximize the usage of all available resources.

Performance Upgrade and Big Savings

The result was a remarkable improvement in runtime from two hours and 20 minutes to just 37 minutes, which was a 400% acceleration. Not only was the performance level upgraded, but high value for cost was also achieved with hardware estimated cost of 3000 Euros and zero software cost. Aside from the glaring benefits in numbers, the experience of using Hadoop and its related technologies have also armed the customer with the knowledge that Hadoop works even for medium-sized data.

ADASTRA GmbH will be attending our Business Analytics, Germany event on the 6th of April 2017 as a solution provider. For all upcoming events, check out our Event Calendar »

The Most Efficient Working Day!

Invited Guests

The most efficient working day. Insights and ideas from the stage and from a network of executives. The best solutions for your business challenges.

Read more »

Solution Providers

Meet executives with investment needs. Bring your solutions and insights to your most potential clients. 11 markets, 20 000 executives, guaranteed meetings.

Read more »

Management Events brings together top-level executives and solution providers, providing high value to both parties. Our concept attracts 20 000 visionary leaders to our events in eleven countries, over 170 times a year. Management Events Surveys provides insights and trends for solution providers and executives, helping them gain deeper understanding of challenges and needs of the largest corporations.

Sorry but your browser screen is too small for this site.