Advancing OAM-Based FSO Systems: Tackling Pointing Errors for Next-Generation Space and Terrestrial Links

Orbital Angular Momentum (OAM)-based Free-Space Optical (FSO) communication systems offer immense potential for high-capacity, secure links, but their performance is highly sensitive to pointing errors. This paper provides a comprehensive analysis of OAM systems under pointing inaccuracies for both short-range terrestrial links and long-range inter-satellite communications. For short links, we demonstrate the trade-offs between increasing modulation order and the number of OAM modes, highlighting the potential of low-complexity two-stage detectors to mitigate computational overhead while maintaining robust performance. For inter-satellite links, where severe pointing errors dominate, we propose an innovative asymmetric mode design and optimal beam waist adjustment to enhance robustness against angular misalignments. Simulation results reveal critical insights into the interplay between pointing error intensity, mode selection, and system performance, offering practical guidelines for OAM system design in diverse scenarios. This work bridges the gap between theoretical modeling and real-world implementation, paving the way for reliable OAM-based communication in next-generation terrestrial and space networks.