For our Research and Development (R&D) activities in the field of Big Data, Machine Learning and Video Analysis Catalysts started up a cluster in the Catalysts office in Linz.
As Cluster Management Software Cloudera is in operation.
Hadoop is used for Big Data Batch Processing.
Storm is used for Big Data Real Time Processing.
The cluster operates with 176 CPU cores, 208 GB RAM and 22 SSDs (in total 4 TB).
Additionally 5.500 CUDA GPU Accelerator cores at 1.63 GHz with 28 GB RAM do mainly count to the 12 TFlop theoretical performance.
250 GByte per minute theoretical data streaming performance should be a convenient value for daily work in software development.
The cluster is up since 2013 February 18 and replaces our EC2 cluster. Practical benchmarks with zipped data showed, that the cluster is able to stream 70 GByte per minute. We hope that within one month we are able to double this value.
Worker Node Data: All nodes of the cluster are crossbar interconnected by a 10 GBit/s network.
- CPU: 2,2 GHz i7-2670QM (8 hypercores)
- 8 GB RAM
- 160 GB SSD
- 250 CUDA GPU Accelerator cores (in average)
- Operating System: Ubuntu 12.10
Catalysts uses the new cluster for Development
- We now have in-house cluster capacity to run and test our big data development for various projects
- We run performance test to increase the throughput of the cluster on a distinct problem
- We are interested in how to reduce a data-set and after reduction it still contains all equivalence classes for correct testing
- Big Data Real Time processing
Interested in Big Data and/or High Performance Computing? Make an appointment