Forward and Inverse Kinematics Solution of A 3-DOF Articulated Robotic Manipulator Using Artificial Neural Network

ABSTRACT


Introduction
In the research literature review [1]- [3], the positions and orientations of the end-effector in the forward kinematics problem are determined using the joints' variables. This process is straightforward, easy, and not complex. This part is very important and crucial for calculating the error of the position and/or the orientation to calculate the controller's qualify [4]. In contrast, inverse kinematics is opposite to forward kinematics in such a way that the joints' variables can be calculated using known positions and orientations of the robot end-effector. This process is complex, difficult, For the inverse kinematics problem, also different types of solutions were considered by researchers, such as 1) closed-form solution method, 2) numerical solution method, 3) evolutionary computing, and 4) neural networks (NNs), as shown in Fig. 2. Closed-form solution method was used when the analytic expression or the polynomial has less than 4-DOF [2]. This solution depended on robot-specific geometry for the formulation of the mathematical model. An example of this approach was presented in [20], where Lou et al. used the closed-form solution for the inverse kinematics of the manipulator based on the general spherical joint. Numerical methods were proposed for the kinematics problem in the inverse mode in case of the resulting polynomial in the solution has more than 4-DOF [21]. These mentioned methods are less accurate compared to the methods of closed-form [2]. An example of these methods was presented in [22], where Elnady developed an iterative technique to solve the inverse kinematics. Evolutionary computing was proposed in ref. [23], [24] as a method for inverse kinematics solution. NNs were also developed to solve the problem of inverse kinematics. Tejomurtula and Kak [25] presented the structured NNs for inverse kinematics. The conventional backpropagation algorithm was performed for the NN training. This led to some difficulties regarding accuracy. NNs were also used in ref. [3], [26], [27]. From the above discussion, further investigation is recommendable for using the NN as a simple and intelligent technique compared with other methods for forward and particularly inverse kinematics solutions. Achieving high levels of performance and reliability of the NN is required by having small values of mean squared error (MSE), which are close to zero and the training error. This can increase the accuracy and robustness of the application and estimate the forward and inverse kinematics correctly. In addition, minimizing the input size of the implemented NN is also recommendable to minimize the complexity and mathematical computations.
The primary contribution of our research paper is proposing an MLFFNN network to solve the issue of kinematics for the 3-DOF articulated robotic manipulator in both forward and inverse modes. In forward kinematics used in the design of the network, the positions and orientations for the manipulator end-effector are outputs, and the joints' variables for the manipulator are used as the inputs. For the inverse kinematics case, only the positions of the end-effector are used as the inputs of the implemented MLFFNN to minimize the size of inputs. The joints' variables are the outputs. In each case, the proposed network is trained using LM learning, which provides fast convergence easily. The main concern during the training on the MLFFNN is obtaining a very small (close to zero) MSE and training error. The training is executed in MATLAB using collected data considering the sinusoidal joints' motion of the manipulator. The testing and the verification of the trained MLFFNN are presented to investigate its reliability in minimizing the approximation error and its effectiveness in estimating the forward and inverse kinematics correctly.
The rest of this paper is divided into the following sections. Section 2 illustrates the forward kinematics of the 3-DOF articulated robot using DH parameters and the inverse kinematics depending on the geometrical approach. In Section 3, the collected data considering sinusoidal joints' motion are described. The design, analysis, and testing of the proposed network to resolve the kinematics problem in forward mode are explained in Section 4. In Section 5, the architectural concept, training, testing and verification of the proposed MLFFNN for solving the problem of inverse kinematics are presented. Finally, Section 6 summarizes the work shown in this manuscript, and Section 7 gives some points for work in the future.

Kinematics of a 3-DOF Articulated Manipulator in the Forward and Reverse Mode
This section discusses the forward kinematics as well as the inverse kinematics of the 3-DOF articulated (with rotational joints) manipulator. This type of robot is presented in Fig. 3. In Subsection 2.1, parameters of Denavit-Hartenberg (DH) are used to illustrate the concept of forward kinematics, whereas a geometrical method is used to show the concept of inverse kinematics in Subsection 2.2. Fig. 3. The 3-DOF articulated robotic manipulator [28], [29]

Forward Kinematics
The forward kinematics of the 3-DOF manipulator is performed using the DH parameters, which are considered the standard for robot kinematics description. These parameters are presented in Table  1. In the table, is the link length, is the link twist, is the link offset, and is the joint angle. According to Table 1, the position and orientation of the robot end-effector are represented by a whole homogeneous transformation matrix, which is obtained as Equation (1) [8].

Inverse Kinematics
In this subsection, the inverse kinematics (the joints' variables 1 , 2 , and 3 ) are determined using the geometrical approach. This approach solves the joint variable by projecting the robot onto the plane of −1 − −1 and then solving a simple problem of trigonometry.
where, 2 is defined only when 2 + 2 + ( − 1) 2 > 0. It is clear from equations (2)-(4) the solution of inverse kinematics, depending on the geometrical approach, is complex and more cumbersome. Furthermore, this is familiar in robotics books that the inverse kinematics solution is complex. Therefore, an MLFFNN network is used for solving the kinematics of the 3-DOF articulated manipulator in forward and inverse. MLFFNN has the following properties and advantages: 1) It is a very simple architecture in comparison to the other different NNs types [13], [30], [31].
2) It has the ability of adaptivity, parallelism, and generalization [32]- [34]. In addition, it can be linear or nonlinear.
However, the MLFFNN network needs a high number of input and target pairs in the training stage [41], [42]. This disadvantage is taken into consideration in the current research, and we use large datasets. However, overfitting is avoided, as seen from the results.
In this current research work, the developed MLFFNN is trained depending on the Levenberg-Marquardt (LM) learning algorithm, which possesses the following qualities: 1) The work can be implemented quickly with the mentioned learning algorithm. It is an optimization technique with a second order that approximates Newton's Method and has a solid theoretical foundation as well as quick convergence [43], [44].
2) This algorithm is preferred because it strikes a balance between the gradient descent algorithm's assured convergence and the quick learning speed of the traditional Newton's method [43], [45]. In addition, its preference for large datasets and its convergence in fewer iterations and a short time in comparison with the other learning methods [31].
Using the LM algorithm, the adjustment of the weight ∆ , which is applied to the parameter vector , is calculated using Equation (5) [13], [31], where and represent, respectively, the Hessian and the gradient vector of the second-order function. is the identity matrix which, its dimension is the same dimensions as . is a regularizing parameter responsible for forcing the part ( + ) to be positive definite as well as safely wellconditioned through the computation.
The next section discusses in detail the generated data that have been used for the analysis, testing, and validation of the MLFFNN network.

Generated Data
This section describes the data that have been utilized for validating and testing the proposed implemented MLFFNN network to solve the forward and inverse kinematics. A sinusoidal motion is chosen to be commanded to the three joints of the articulated robotic manipulator. This motion is given by Equation (6).
where is the joint 1, 2, and 3, is the frequency of the sinusoidal type of motion, and it is variable and linearly increasing.
This type of kinematic is similar to the motion presented in our previous work [46]. The range of each joint motion is [−90, 90] . The position and the orientation of the robot end-effector is calculated according to Equation (1)

Forward Kinematics Solution Using MLFFNN
This section presents designing, training, testing, and verifying the proposed MLFFNN to solve the forward kinematics of a 3-DOF articulated manipulator.

Network Design
The proposed MLFFNN is designed using three layers. The first layer is the input layer which contains the three inputs, which are the joints' variables 1 , 2 , and 3 . The hyperbolic tangent, denoted by tanh, serves as the activation function for the second layer, which is the hidden, non-linear layer. It also contains hidden neurons. The output layer is the third layer. This layer is linear and estimates the values of the position and the orientations for the targeted robot ( ′ , ′ , ′ , 11 ′ , 12 ′ , 13 ′ ……. 33 ′ ), which are twelve outputs. These estimated outputs are compared with the actual positions and orientations (which are presented in Fig. 5 and Fig. 6). The main and followed criteria during the design of the MLFFNN-network is implementing a simple NN can achieve high levels of performance which can be represented by small mean squared error and training error. In other meaning, these values should be close to zero value. The proposed MLFFNN is presented in Fig. 7. The intended end-effector positions and orientations of the robot are used only for the training of the designed MLFFNN. In addition, the error resulting from training must be small as possible and close to zero. The training process is described in the following subsection.

Input Layer
Hidden Layer

Designed Network Training
The collected data that are presented in section 3 are used for analyzing the behavior of the proposed MLFFNN network. These data are divided into three groups as follows: 1) 80% of these data (24315 samples) are used for the training, 2) 10% (3040 samples) are used for validation, and 3) The last 10% (3040 samples) are used for testing.
This division in data occurs in MATLAB randomly. In addition, this is very important to be able to test and validate the trained NN by different types of data used for the training procedure. Therefore, we can be sure that the trained NN works effectively and can estimate the output value correctly. The training is occurring in MATLAB using LM learning. The steps and methodology that followed during the training and test processes of the designed MLFFNN network are presented in Fig. 8.
After executing many experiments and trials by using different initialization of weights and different hidden neurons number, the preferred settings of parameters that enable the MLFFNN's high performance are obtained. These parameters are the best number of hidden neurons, the number of the used epochs or iterations, the very small MSE value, and the training time. All these parameters are presented in Table 2. Some other resulting parameters are presented in the Appendix. The intended high performance is achieving the lowest MSE as well as the training error. The time of training is not a very important issue because the training happens offline, and the main aim is to have a very welltrained MLFFNN network that can estimate the outputs efficiently with an error close to zero. The MSE value is calculated using Equation (7), where is the actual output that is used for the training process, and is the estimated output by the designed NN. is the number of samples. The MSE and the regression obtained from the training are shown in Fig. 9 and Fig. 10. The resulting error histogram is presented in the Appendix. As presented in Fig. 9, the obtained MSE is a very small value and approximately zero. In addition, the resulting regression is 1, as shown in Fig.  10. These results illustrate that the required positions and orientations for the targeted robot end effector converge/coincide with the corresponding estimated ones by the MLFFNN network. In other meaning, the training error between them is approximately zero. The simulation results demonstrate that the analyzed MLFFNN network is learned and trained efficiently, which can estimate the positions and the orientations of the robot's end-effector. The process of testing and the verification of the trained method is presented in the following subsection. This process of very important to show the effectiveness and reliability of the method.

Trained Network Verification and Testing
All samples (30395) are used to verify and test the trained MLFFNN. Results and comparisons of the robot's desired effector's positions and orientations, and the corresponding estimated ones by the trained network are presented from Fig. 11 to Fig. 14. Furthermore, the average, the maximum, the minimum, and the standard deviation (std) of the approximation error between the desired and estimated end-effector positions are shown in Table 3.
From Fig. 11, Fig. 12, Fig. 13 and Table 3, it is clear that desired positions of the robot endeffector in −, −, and − directions and the corresponding estimated ones by the NN are coincide. The approximation error between them is a very small value and approximately zero. In Fig. 14, the desired orientations of the robot also coincide with the corresponding approximated ones by the neural network. This also means that the approximation error is a very small value and about the zero value. These results prove that the trained MLFFNN is highly reliable in minimizing errors and is trained very well. Furthermore, it is able to solve the forward kinematics (finding positions and orientations of end-effector) of a 3-DOF articulated manipulator in a correct way.

Inverse Kinematics Solution Using MLFFNN
This section shows in detail the designing, training, testing, and verifying of the proposed implemented MLFFNN to resolve the 3-DOF articulated robotic manipulator's inverse kinematics. Fig. 15 shows his design of MLFFNN.

Network Design
The proposed MLFFNN network is designed in such a way that using three layers of neurons are interconnected with each other. The first layer is called the input layer, which contains the three inputs, which are only the positions of the robot's end-effector ( , , and ). The second layer, which contains hidden neurons, is called the hidden layer, which is considered non-linear in its operation, and the activation function for this layer is the hyperbolic tangent which is represented by tanh. The third and last layer is the output layer. This layer is linear and estimates the joints' variables ( 1 ′ , 2 ′ , and 3 ′ ), which are three outputs. These estimated outputs are compared with the actual (desired) joints' variables (which are presented in Fig. 4). This proposed MLFFNN is presented in Fig. 15.
The desired joints' variables are utilized for training the designed MLFFNN network. The error from the training phase must be very small and close to the zero value. In detail, training for the implemented MLFFNN is described in the following subsection (

Designed Network Training
The same procedure that is presented in Subsection 4.2 is followed here. The collected data mentioned in Section 3 is divided as follows: 1) 80% of the collected data (24315 samples) are used for the training, 2) 10% (3040 samples) are used for validation, and 3) The last 10% (3040 samples) are used for testing.
The most suitable parameters which lead to high levels of performance for the designed MLFFNN network are the best number of hidden numbers, the number of epochs or iterations, the very small MSE, and the training time. These parameters are presented in Table 4. As mentioned in Section 4, the training time is not very important. Some other parameters are presented in the Appendix. The resulting MSE and regression from the training are shown in Fig. 16 and Fig. 17. The obtained error histogram is presented in the Appendix. As shown from the figures, the MSE is very small and approximately zero. The regression is equal to 1. This means that the desired joints' variables are coinciding/converging with corresponding estimated ones by the implemented NN. Therefore, the training error is very small. This indicates that the designed MLFFNN-neural network is trained efficiently and it is ready to solve inverse kinematics correctly. Testing and the verification of the trained NN should be investigated, as presented in the following section. This process is very crucial for showing the effectiveness of the trained method.

Trained Network Verification and Testing
In this step, all samples (30395) are used to verify and test the trained MLFFNN. The results and the comparisons between the targeted joints' variables and the corresponding computed ones by the trained MLFFNN-neural network are presented in Fig. 18, Fig. 19, and Fig. 20. Furthermore, Table 5 presents the average, maximum, minimum, and standard deviation of the approximation absolute error between the desired and estimated joints' variables. It is clear from the presented figures (Fig. 18, Fig. 19, and Fig. 20) and Table 5 that the desired joints' variables (Theta1, Theta 2, and Theta 3) coincide with the corresponding estimated ones by the NN. The approximation error between them is a very small value and approximately zero value. This proves that the trained MLFFNN is highly reliable in minimizing error and is trained very well. It should be noted that the trained MLFFNN takes 0.587 seconds in MATLAB to do all the calculations and give the results. This time is very short, and if the method is applied experimentally/practically to any 3-DOF robot, it cannot affect the continuity of the robot' motion. The time in the case of the inverse kinematics is less than the time in the forward kinematics due to the fact that MLFFNN-network estimates 12 outputs in the forward kinematics case, and it estimates only 3 outputs in the other case.

Remark:
To solve the inverse kinematics, we created another MLFFNN and used both the positions and orientations of the robot end-effector as its inputs. The results are better compared with the presented one in this section (Section 5). However, the presented MLFFNN in this section (Section 5) gives very good results because the MSE and training/approximation error are both very low values that are close to zero. In addition, we prefer it since the inputs are smaller than when using the endeffector locations and orientations. The complexity of the presented MLFFNN is also lower.

Limitation of the proposed method:
The proposed method is investigated using a limited range of the joints' motion of the manipulator as the range of each joint motion is [−90, 90] . Therefore, applying the method using the entire experiment space of the robot joints should be investigated and considered in future work. The proposed method is used with a 3-DOF articulated robot. For investigating the generalization of the proposed network, it should be applied to other different types of robots and more complex robots like 6-DOF robots or 7-DOF robots.
In the current work, only sinusoidal motion is recommended for the robot joints. Other different types of joint motion should also be considered and used. The proposed method is validated only using simulated data, and no experimental data or real robot experiments are conducted to validate the effectiveness of the proposed method in real-world scenarios. This happens because we do not have a real robot at the time of doing this paper. However, in future work, the experimental validation should be considered and investigated.
The accuracy of the proposed MLFFNN for solving the problem of a 3-DOF manipulator working in the inverse kinematics is compared with other previous NNs-based approaches such as the ones presented by Koker et al. [47], Daya et al. [48], Duka [3], and Jiménez-López et al. [49]. With Duka [3], a feedforward NN was used for solving the inverse kinematics of a 3-DOF planar manipulator. The NN was designed using six inputs which were the three end-effector positions and the three orientations. With Koker et al. [47], a backpropagation NN with a sigmoidal activation function was proposed as a foundation for the inverse motion problem for the 3-DOF manipulator robot. In [48], Daya et al. developed an NN-based approach for the inverse kinematics solution of a 3-DOF manipulator.
In their design, six inputs were used, which were the three positions and the three orientations of the end-effector. Jiménez-López et al. [49] implemented a NN for the inverse kinematic solution of a 3-DOF manipulator. In their approach, the NN was trained using Bayesian regularization backpropagation. In addition, the limited range of the joints' motion of the manipulator was considered. The comparison includes the resulting MSE, which is the main parameter to show the accuracy of each method. The comparison is shown in Fig. 21. As shown in Fig. 21, the proposed method has the resulting smallest MSE value compared with other previous related approaches. Hence, the results prove that the proposed method is the most accurate, and the resulting approximation error is the smallest.

Conclusion
In this research, the kinematics of a 3-DOF articulated robotic manipulator in both forward and inverse modes are solved using the concept of the MLFFNN-network algorithm. For Forward kinematics, the MLFFNN is designed using the joints' variables as its inputs. In addition, its outputs are the positions and orientations of the end-effector of the robot. For the kinematics in the inverse case, only the positions of the end-effector are used as the inputs of the MLFFNN. The outputs are the joints' variables. For both cases, the designed MLFFNN network is trained using the LM learning algorithm using data collected from the sinusoidal joint motion of the manipulator. The resulting training errors and the MSE from the training stage have relatively very small values and close to zero values. The simulation results demonstrate that the proposed neural network is trained in an effective way. The trained MLFFNN is tested and verified, and the results prove that the approximation error between the actual output and the corresponding predicted one by the NN is equal to a very small value. Therefore, the MLFFNN is highly reliable in minimizing errors. Furthermore, it is efficient to solve the kinematics of the robot in forward and inverse modes correctly. Our proposed research approach is compared with other previously published methods. The experimental simulation result reveals that our proposed neural network-based method has the highest levels of accuracy compared with others.

Future Work
Different aspects of future work can be discussed in this section. Firstly, as the range of each joint motion of the used robot is limited in the current work and is [−90, 90] , the future work can consider the whole workspace of the robot joints. Secondly, the use of the NN in solving the forward and inverse kinematics of a complex robot, such as a 6-DOF or a 7-DOF manipulator, can also be considered. Thirdly, different types of NNs-based architectures inspired by biological concepts can be used and compared, such as WaveNet, RNN, and RBF. Deep learning methods can also be investigated and compared. Fourthly, we would synthesize the hardware implementation for the proposed MLFFNN network on file programmable gate array (FPGA) technology and use different fault-tolerant techniques. FPGA has many advantages, as follows: 1) It can easily change its functionality after designing the product.
2) It does not require a larger board area, and it is more energy efficient than other equivalent discrete circuits. 3) It can carry out several operations on data simultaneously.
Finally, investigating the current work experimentally with a real robot is recommended. Author Contribution: All authors contributed to this paper. Most of the presented work is done by Abdel-Nasser Sharkawy. All authors read and approved the final paper.