Big Data Processing System Optimization for Digital Healthcare Based on Hadoop and Spark Architecture

Abstract

The rapid growth of data in digital healthcare demands efficient, fast, and scalable processing systems. Data from Electronic Health Records (EHRs), Internet of Medical Things (IoMT) devices, and telemedicine services generate massive and complex volumes of information. This research aims to optimize big data processing by utilizing Hadoop and Spark integrated architectures in hospital information systems. The method used is a qualitative approach with in-depth interview techniques, observations, and questionnaires to the information technology and hospital management teams. The research was conducted in two private hospitals that have implemented a comprehensive digital system. The results show that the integration of Hadoop as a distributed storage system with Spark as an in-memory processing engine can increase operational efficiency by up to 47%, reduce execution time by up to 60%, and provide more stable performance than conventional methods such as MapReduce. Data visualization supports this claim with a significant comparison of runtime and resource usage. These findings imply that the Hadoop–Spark architecture is a strategic solution for real-time and batch processing of health data. This research also offers an application model that can be replicated in other health institutions in Indonesia.