Real-time Analytics on Apache Spark Based Cloud Computing System

Graduate #58
Discipline: Computer Sciences and Information Management
Subcategory: Computer Science & Information Systems

Waled Tayib - Tennessee State University

---

Recently, the cloud computing industry has seen tremendous growth due to the widespread use of cloud resources for the Internet of Things (IoT). The increased use of IoT means that more and more data will be generated from sensors, machines, software, etc. With the amount of data generated daily being in the terabytes, there will be a higher demand for real-time processing capabilities in the cloud. A study conducted by the National Science Foundation in 2015 shows that IoT is on track to connect 50 billion devices by 2020, and one trillion sensors soon after. This research project presents a cloud computing infrastructure that has been set up at Tennessee State University to process all this data coming from IoT in real-time. The processing infrastructure consists of an Apache Spark based cloud-computing platform to ensure fast response. First, the performance of the Spark cluster was examined after conducting various real-time jobs such as machine learning, searching, and event counting. After testing, performance of the system has shown to achieve a general response time of less than one second. Future research on this project will consist of optimizing the Spark cluster specifically for each type of real-time processing use-case.

Not Submitted

Funder Acknowledgement(s): This study was funded by the NSF Research Initiation Award awarded to Dr. Sachin Shetty, Assistant Professor, Electrical and Computer Engineering, Tennessee State University, Nashville, TN 37209.

Faculty Advisor: Sachin Shetty, sshetty@tnstate.edu