Forest fires represent a significant environmental threat, primarily due to the substantial economic losses and human casualties they cause. Smoke serves as the most apparent early indicator of wildfires. However, detecting smoke effectively with computer vision is challenging, particularly when it is distant from the camera and only noticeable in a limited area. Overcoming this challenge with current methodologies often necessitates computationally intensive processing of higher resolution images or the installation of a higher density of cameras, both representing costly solutions. To address these limitations, we introduce a novel framework called Video-based Multiple Objects Kinetic Emission Detection (VMOKED). It is specifically designed to detect smoke emissions in forests accurately and scalably, even when operating under constrained computational budgets in centralized multi-stream processing environments. The VMOKED framework integrates three key components: (1) a real-time object detection model, (2) a compact Sky-Ground Segmentation (SGS) model utilizing a feature pyramid network architecture and MobileNetV3 backbone, and (3) a motion measurement module designed to leverage temporal information efficiently. Experimental results on our dataset demonstrate that the proposed VMOKED framework surpasses other methods in quantitative performance. Highlighting its efficiency, VMOKED requires only 64.89 Giga Floating-Point Operations Per Second (GFLOPs) and achieves a detection time of just 7.65 ms.