This paper investigates hybrid free-space optics (FSO)/radio frequency aerial access networks (AANs) using a high-altitude platform (HAP) and multiple UAVs to dynamically serve terrestrial users under varying environmental conditions, such as atmospheric turbulence and cloud-induced attenuation. The optical intelligent reflecting surfaces (OIRS), mounted on the HAP, enhance the FSO signal distribution to multiple UAVs by enabling precise beam manipulation, improving link reliability, and increasing network scalability. A deep reinforcement learning (DRL)-based approach is developed to optimize UAV placement and user association in real time, maximizing end-to-end throughput while adhering to backhaul capacity constraints. The study takes into account FSO channel impairments, including path loss, turbulence-induced fading, and pointing misalignment, modeled using log-normal distributions. Numerical results demonstrate that the dynamic deployment of a multi-UAV configuration, trained under realistic cloudy conditions, significantly outperforms single-UAV and static deployment strategies, achieving higher data rates and stable user connectivity. This work highlights the potential of deploying OIRS-assisted AANs supporting multiple UAVs to realize robust and high-performance 6G networks.