Non-terrestrial networks (NTNs), incorporating unmanned aerial vehicles (UAVs) and optical base stations (OBSs), are emerging as a key architectural component for the Sixth Generation (6G) wireless networks, particularly in supporting a wide range of vehicular services in the Internet of Vehicles (IoV). In parallel, free-space optical (FSO) and sub-terahertz (sub-THz) communications are expected to play a central role in 6G due to their abundant spectrum resources. However, the deployment of UAV-assisted vehicular services faces significant challenges, as highly dynamic vehicle mobility and time-varying atmospheric turbulence introduce severe link fluctuations and reliability issues. A hybrid FSO/sub-THz communication link is a potential solution to this problem by enhancing link reliability and service continuity, but it necessitates an intelligent and adaptive switching mechanism. To address this, we propose a deep reinforcement learning (DRL) algorithm that integrates both hard-switching and soft-switching mechanisms to guarantee the UAV-supported quality-of-service (QoS) through real-time trajectory and link optimization. The hybrid FSO/sub-THz link is modeled by considering atmospheric attenuation, fading, pointing errors, blockage by buildings, and weather conditions. Simulation results indicate that the trained UAV can effectively learn from the dynamic environment and maintain robust performance using the proposed DRL-based approach.