Data analysis service

Hadoop-based data analysis service.


Data analysis service or PARADOX Hadoop cluster consists of a single name node that runs the YARN resource manager, and three additional data nodes. The name node is hosted on a machine with 4-core Intel Xeon E3-1220v3 CPU running at 3.1 GHz, with 4 GB of RAM, and 500 GB of local hard disk storage. Each of the data nodes, which perform the computation and storage, are hosted on machines with 24-core Intel Xeon E5-2620 CPUs at 2.4 GHz, with 64 GB of RAM and 2 TB of storage. In total, the cluster provides access to 60 CPU cores, 180 GB of RAM and 5.3 TB of storage in HDFS.

In the analysis of very large datasets, the movement of data can present a far more severe bottleneck than the actual computation. Therefore, the PARADOX Hadoop cluster is designed to overlap computation and data storage operations, i.e., to enable performing of computation on the same machine(s) that store the corresponding data.

Resource organization

Resource providers

ON-BOARDED 
...
Institute of Physics Belgrade

Pregrevica 118, 11080 Belgrade, Serbia (RS)

ON-BOARDED 
...
Institute of Physics Belgrade

Pregrevica 118, 11080 Belgrade, Serbia (RS)

 GENERIC
ON-BOARDED 

...

Website

https://www.scl.rs/PARADOX_User_Guide/hadoop-user-guide.html

Readiness

TRL9

Domain

Natural Sciences Engineering & Technology

Subdomain

Chemical Sciences Physical Sciences Biological Sciences Environmental Biotechnology Mathematics Other Natural Sciences Materials Engineering Chemical Engineering Computer & Information Sciences Environmental Engineering Industrial Biotechnology Nanotechnology Mechanical Engineering Earth & Related Environmental Sciences

Category

Data Analysis Compute

Subcategory

2D/3D Digitisation Job Execution Machine Learning Image/Data Analysis Forecast Data extrapolation Visualization Artificial Intelligence Workflows

Target users

Researchers

Access type

Virtual

Access mode

Peer reviewed

Tags

hadoop cluster big data

Contact

Dusan Vudragovic, Assistant Professor
email: dusan@ipb.ac.rs
phone: +38163699621

Helpdesk e-mail

hpc-admin@ipb.ac.rs

Security contact e-mail

hpc-admin@ipb.ac.rs

The NI4OS-Europe project is funded by the European Commission under the Horizon 2020 European research infrastructures grant agreement no. 857645.