Download Free Reinforcement Learning Framework For Spacecraft Low Thrust Orbit Raising Book in PDF and EPUB Free Download. You can read online Reinforcement Learning Framework For Spacecraft Low Thrust Orbit Raising and write the review.

The use of electric propulsion (EP) in satellites for transfer to geosynchronous equatorial orbit (GEO) is increasingly gaining importance among the space industry all around the world, and is proven a key for new space missions. In a conventional launch, the satellite is placed into a geostationary transfer orbit (GTO) by the launch vehicle and uses chemical propellants to reach GEO. This orbital transfer maneuver typically takes a few days. However, even though EP is far more e cient than the conventional chemical propulsion, its low thrust generation adds the complexity of longer transfer time from an equatorial orbit to GEO. This longer transit time leads to exposure of spacecraft to hazardous radiation of Van Allen belts. Therefore, there is a need to develop a method to determine the minimum transfer time trajectory for all-electric low thrust orbit raising problem. This thesis proposes a new formulation that facilitates the application of reinforcement learning to the problem of orbit raising. This work is based on the approach that the electric orbit-raising problem is posed as a sequence of multiple trajectory optimization sub-problems. Each sub-problem aims to move the spacecraft closest to GEO by minimizing a convex combination of suitably selected objectives. A mathematical formulation for the orbit-raising problem is proposed in the framework of reinforcement learning to enable adaptive modi cation of the objective function weights during a transfer. Due to high dimensionality of the planning states of the orbit-raising problem, arti cial neural networks are then constructed and trained on orbit-raising scenarios in order to compute the reward functions associated with reinforcement learning. The reward function for a planning state is de ned as the time required to reach GEO from that planning state. With the help of numerical simulations for planar and non-planar transfer scenarios, it is demonstrated that there is a reduction in transfer time for low-thrust orbit raising problem with the proposed methodology.
This dissertation explores a novel method of solving low-thrust spacecraft targeting problems using reinforcement learning. A reinforcement learning algorithm based on Deep Deterministic Policy Gradients was developed to solve low-thrust trajectory optimization problems. The algorithm consists of two neural networks, an actor network and a critic network. The actor approximates a thrust magnitude given the current spacecraft state expressed as a set of orbital elements. The critic network evaluates the action taken by the actor based on the state and action taken. Three different types of trajectory problems were solved, a generalized orbit change maneuver, a semimajor axis change maneuver, and an inclination change maneuver. When training the algorithm in a simulated space environment, it was able to solve both the generalized orbit change and semimajor axis change maneuvers with no prior knowledge of the environment’s dynamics. The robustness of the algorithm was tested on an inclination change maneuver with a randomized set of initial states. After training, the algorithm was able to successfully generalize and solve new inclination changes that it has not seen before. This method has potential future applications in developing more complex low-thrust maneuvers or real-time autonomous spaceflight control.
Artificial intelligence is a rapidly developing field that promises to revolutionize spaceflight with greater robotic autonomy and innovative decision making. However, it remains to be determined which applications are best addressed using this new technology. In the coming decades, future spacecraft will be required to possess autonomous guidance and control in the complex, nonlinear dynamical regimes of cis-lunar space. In the realm of trajectory design, current methods struggle with local minima, and searching large solutions spaces. This thesis investigates the use of the Reinforcement Learning (RL) algorithm Proximal Policy Optimization (PPO) for solving low-thrust spacecraft guidance and control problems. First, an agent is trained to complete a 302 day mass-optimal low-thrust transfer between the Earth and Mars. This is accomplished while only providing the agent with information regarding its own state and that of Mars. By comparing these results to those generated by the Evolutionary Mission Trajectory Generator (EMTG), the optimality of the trajectory designed using PPO is assessed. Next, an agent is trained as an onboard regulator capable of correcting state errors and following pre-calculated transfers between libration point orbits. The feasibility of this method is examined by evaluating the agent’s ability to correct varying levels of initial state error via Monte Carlo testing. The generalizability of the agent’s control solution is appraised on three similar transfers of increasing difficulty not seen during the training process. The results show both the promise of the proposed PPO methodology and its limitations, which are discussed.
Recently, there has been a surge in use of electric propulsion to transfer satellites to the geostationary Earth orbit (GEO). Traditionally, the transfer times to reach GEO using all electric propulsion are obtained by solving challenging trajectory optimization problems, whose solution rely on numerical schemes that are not only computationally intensive, but also lack automated implementation capabilities. This naturally creates a hindrance to their incorporation within Deep Reinforcement Learning (DRL) framework, which combines Reinforcement Learning (RL) and Deep Learning to solve trajectory planning problems in near real-time. The operation of DRL, as typically used in trajectory planning, relies on a Q-value. In the electric orbit-raising problem under consideration in this thesis, this Q-Value requires computation of transfer time in near real-time to have practical DRL training times. In our work, this Q-value is predicted by a set of deep neural networks (DNNs) instead of solving traditional optimization problems. This thesis aims at designing a set of DNNs that can serve as a Q-value (transfer time) predictor for different orbit-raising mission scenarios. To this end, we investigate different architectures for DNNs to determine the optimal DNN configuration that can predict the transfer time for each of the mission scenarios. Experimental results indicate that our designed DNNs can predict the transfer time for different scenarios with an accuracy of over 99%. We also compare the results from our designed DNNs with the contemporary Machine Learning (ML) algorithms, such as Support Vector Machine (SVM), Random Forest (RF), Decision Tree (DT) for regression. Experimental results indicate that our best-performing DNNs can provide an improvement in mean error of transfer time prediction of up to 14.05x for non-planar transfers and up to 254x for planar transfers.
All-electric satellites are gaining favor among the manufacturers and operators of satellites in Geostationary Earth Orbit (GEO) due to cost saving potential. These satellites have the capability of performing all propulsive tasks with electric propulsion including transfer to GEO. Although fuel-efficient, electric thrusters lead to long transfer, during which the health and the usability of spacecraft is affected due to its exposure to hazardous space radiation in the Van Allen belts. Hence, determining electric orbit-raising trajectory that minimize transfer time is crucial for all-electric satellite operation. This thesis proposes a novel method to determine minimum-time orbit-raising trajectory by blending the ideas of direct optimization and guidance-like trajectory optimization schemes. The proposed methodology is applicable for both planar and non-planar transfers and for transfers starting from arbitrary circular and elliptic orbits. Therefore, it can be used for rapidly analyzing various orbit-raising mission scenarios. The methodology utilizes the variational equations of motion of the satellite in the context of the two-body problem by considering the low-thrust of an electric engine as a perturbing force. The no-thrust condition due to Earth's shadow is also considered. The proposed methodology breaks the overall optimization problem into multiple sub-problems and each sub-problem minimizes a desired objective over the sun-lit part of the trajectory. Two different objective types are considered. Type I transfers minimize the deviation of the total energy and eccentricity of final position from the GEO, while type II transfers minimize the deviation of total energy and angular momentum. Using the developed tool, several mission scenarios are analyzed including, a new type of mission scenarios, in which more than one thruster type are used for the transfer. The thesis presents the result for all studied scenarios and compares the performance of Type I and Type II transfers.
The evolution of low-thrust propulsion technologies has reached a point where such systems have become an economical option for many space missions. The development of efficient, low trip time control laws has received an increasing amount of attention in recent years, though few studies have examined the subject of inclination changing maneuvers in detail. A method for performing economical inclination changes through the use of an efficiency factor is derived front Lagrange's planetary equations. The efficiency factor can be used to regulate propellant expenditure at the expense of trip time. Such a method can be used for discontinuous-thrust transfers that offer reduced propellant masses and trip-times in comparison to continuous thrust transfers, while utilizing thrusters that operate at a lower specific impulse. Performance comparisons of transfers utilizing this approach with continuous-thrust transfers are generated through trajectory simulation and are presented in this paper. Falck, Robert and Gefert, Leon Glenn Research Center NASA/TM-2002-211871, NAS 1.15:211871, E-13553, AIAA Paper 2002-4895
The direct approach for trajectory optimization was found to be very robust. For most problems, solutions were obtained even with poor initial guesses for the controls. The direct approach was also found to be only slightly less accurate than the indirect methods found in the literature (within 0.7%). The present study investigates minimum-time and minimum-fuel low-thrust trajectory problems via a single shooting direct method. Various Earth-based and interplanetary case studies have been examined and have yielded good agreement with similar cases in the literature. Furthermore, new near-optimal trajectory problems have been successfully solved. A multiple-orbit thrust parameterization strategy was also developed to solve near-optimal very-low-thrust Earth-based transfers. Lastly, this thesis examines the use of the high-accuracy complex-step derivative approximation method for solving low-thrust transfer problems. For certain very nonlinear transfer problems, the complex-step derivative approximation was found to increase the robustness of the single shooting direct method.
This book presents advanced case studies that address a range of important issues arising in space engineering. An overview of challenging operational scenarios is presented, with an in-depth exposition of related mathematical modeling, algorithmic and numerical solution aspects. The model development and optimization approaches discussed in the book can be extended also towards other application areas. The topics discussed illustrate current research trends and challenges in space engineering as summarized by the following list: • Next Generation Gravity Missions • Continuous-Thrust Trajectories by Evolutionary Neurocontrol • Nonparametric Importance Sampling for Launcher Stage Fallout • Dynamic System Control Dispatch • Optimal Launch Date of Interplanetary Missions • Optimal Topological Design • Evidence-Based Robust Optimization • Interplanetary Trajectory Design by Machine Learning • Real-Time Optimal Control • Optimal Finite Thrust Orbital Transfers • Planning and Scheduling of Multiple Satellite Missions • Trajectory Performance Analysis • Ascent Trajectory and Guidance Optimization • Small Satellite Attitude Determination and Control • Optimized Packings in Space Engineering • Time-Optimal Transfers of All-Electric GEO Satellites Researchers working on space engineering applications will find this work a valuable, practical source of information. Academics, graduate and post-graduate students working in aerospace, engineering, applied mathematics, operations research, and optimal control will find useful information regarding model development and solution techniques, in conjunction with real-world applications.