Uncrewed Aerial Vehicles (UAVs) have garnered significant practical applications in recent years. The use of UAVs, particularly quadrotors equipped with onboard cameras for tasks such as aerial filming, reconnaissance, and search and rescue (SAR) has received growing attention due to their flexibility and superior maneuverability compared to other UAV platforms. With the advancement of AI-powered object tracking algorithms, real-time visual data constitutes a highly promising contribution to quadrotor control systems. Motivated by these trends, this study proposes a lightweight tracking algorithm that is specifically designed for efficient deployment on embedded hardware platforms. The proposed algorithm not only offers a low computational footprint but also improves performance and accuracy in real-time object tracking tasks. In addition, a quadrotor control system capable of following a target in GNSS-denied environments is developed. To validate the effectiveness and deployability of the proposed system, a Software in the Loop (SITL) simulation is conducted, and a Hardware in the Loop (HITL) simulation framework is developed for the Raspberry Pi 5 platform using the PyBullet simulation environment.