Geostationary satellite remote sensing systems are a useful tool for forest fire detection and monitoring because of their high temporal resolution over large areas. In this study, we propose a combined 3-step forest fire detection algorithm (i.e., thresholding, machine learning-based modeling, and post processing) using Himawari-8 geostationary satellite data over South Korea. This threshold-based algorithm filtered the forest fire candidate pixels using adaptive threshold values considering the diurnal cycle and seasonality of forest fires while allowing a high rate of false alarms. The random forest (RF) machine learning model then effectively removed the false alarms from the results of the threshold-based algorithm (overall accuracy ~99.16%, probability of detection (POD) ~93.08%, probability of false detection (POFD) ~0.07%, and 96% reduction of the false alarmed pixels for validation), and the remaining false alarms were removed through post-processing using the forest map. The proposed algorithm was compared to the two existing methods. The proposed algorithm (POD ~ 93%) successfully detected most forest fires, while the others missed many small-scale forest fires (POD ~ 50–60%). More than half of the detected forest fires were detected within 10 min, which is a promising result when the operational real-time monitoring of forest fires using more advanced geostationary satellite sensor data (i.e., with higher spatial and temporal resolutions) is used for rapid response and management of forest fires.