For our Research and Development (R&D) activities in the field of Big Data, Machine Learning and Video Analysis we now have a local cluster in our office.
The cluster operates with 176 CPU cores, 208 GB RAM and 4 TB fast SSD.
Additionally 5.500 CUDA GPU Accelerator cores at 1.63 GHz with 28 GB RAM do mainly count to the 12 TFlop theoretical performance.
Theoretically the cluster is able to stream 250 GByte per minute. For daily work in software development this is a convenient value.
The cluster is now up since 18. Februar 2013 and replaces our EC2 cluster, which will be switched off by today. Practical benchmarks with zipped data showed, that the cluster is able to stream 70 GByte per minute. We think within one month we are able to double this value.
Node Data: All nodes of the cluster are crossbar interconnected by a 10 GBit/s network.
- CPU: 2,2 GHz i7-2670QM (8 hypercores)
- 8 GB RAM
- 160 GB SSD
- 250 CUDA GPU Accelerator cores (in average)
- Operating System: Ubuntu 12.10
As Cluster Management Software Cloudera is in operation.
Hadoop is used for Big Data Batch processing.
Storm is used for Big Data Real Time processing.
Catalysts uses the new cluster for
- We now have in-house capacity to test our big data development for various projects
- We run performance test to increase the througput of the cluster on a distinct problem
- We are interested in how to reduce a dataset and after reduction it still contains all equivalence classes for correct testing
All software developers, testers and data analysts of Catalysts have access to the cluster, which is 24 hours available per day.
Interested in Big Data and/ or High Performance Computing? Make an appointment