Catalysts started up a new R&D Cluster for Big Data

For our Research and Development (R&D) activities in the field of Big Data, Machine Learning and Video Analysis we now have a local cluster in our office.

The cluster operates with 176 CPU cores, 208 GB RAM and 4 TB fast SSD.
Additionally 5.500 CUDA GPU Accelerator cores at 1.63 GHz with 28 GB RAM do mainly count to the 12 TFlop theoretical performance.
Theoretically the cluster is able to stream 250 GByte per minute. For daily work in software development this is a convenient value.

The cluster is now up since 18. Februar 2013 and replaces our EC2 cluster, which will be switched off by today. Practical benchmarks with zipped data showed, that the cluster is able to stream 70 GByte per minute. We think within one month we are able to double this value.

Node Data: All nodes of the cluster are crossbar interconnected by a 10 GBit/s network.

  • CPU: 2,2 GHz i7-2670QM (8 hypercores)
  • 8 GB RAM
  • 160 GB SSD
  • 250 CUDA GPU Accelerator cores (in average)
  • Operating System: Ubuntu 12.10

As Cluster Management Software Cloudera is in operation.

Hadoop is used for Big Data Batch processing.

Storm is used for Big Data Real Time processing.

Catalysts uses the new cluster for

      Development

    • We now have in-house capacity to test our big data development for various projects
    • We run performance test to increase the througput of the cluster on a distinct problem

Research

  • We are interested in how to reduce a dataset and after reduction it still contains all equivalence classes for correct testing

All software developers, testers and data analysts of Catalysts have access to the cluster, which is 24 hours available per day.

Interested in Big Data and/ or High Performance Computing? Make an appointment

Anfrage für Web-Konferenz bzw. Anfrage für persönliches Treffen

 

Previous Post
“Big-Data” Quiz
Next Post
Environment polluters caught by Big Data

Related Posts

No results found

Leave a Reply

Your email address will not be published. Required fields are marked *

Fill out this field
Fill out this field
Please enter a valid email address.
You need to agree with the terms to proceed

Menu