Spiking PID Control Applied in the Van de Vusse Reaction

a Universidad Veracruzana, Prolongación Venustiano Carranza, Col. Revolución, Poza Rica 93390, Veracruz, Mexico b National Engineering School of Gabès, Rue Omar Ibn El Khattab, Zrig Eddakhlania, Gabès 6029, Tunisia c Polytechnic University of Pachuca, Pachuca-Cd. Sahagún Km 20, Rancho Luna, Zempoala, Hgo., 43830, Mexico d Universitas Ahmad Dahlan, Jl. Kapas No.9, Semaki, Kec. Umbulharjo, Yogyakarta, 55166, Indonesia 1 carmarquez@uv.mx; 2 yakoubzaineb@yahoo.fr; 3 marquez@upp.edu.mx; 4 alfian.maarif@te.uad.ac.id * Corresponding Author


Introduction
To work with nonlinear systems, the soft computing techniques are interesting alternatives; one advantage is their capability of learning using data in a similar way to adaptive systems. Soft computing can be seen as a metaphor for the natural world [1]; in particular, Artificial Neural Networks (ANN) try to mimic the connections among some neurons. The first generation of ANN can be described as some switches interconnected to have certain behavior; one example is to decide if is turned on a machine as a logic circuit can make in automation applications [2]. Once the activation function for a neuron was not limited to give 0 or 1 as in Boolean algebra, it is possible to compute the neuron action according to Artificial neural networks (ANN) can approximate signals and give interesting results in pattern recognition; some works use neural networks for control applications. However, biological neurons do not generate similar signals to the obtained by ANN. The spiking neurons are an interesting topic since they simulate the real behavior depicted by biological neurons. This paper employed a spiking neuron to compute a PID control, which is further applied to the Van de Vusse reaction. This reaction, as the inverse pendulum, is a benchmark used to work with systems that has inverse response producing the output to undershoot. One problem is how to code information that the neuron can interpret and decode the peak generated by the neuron to interpret the neuron's behavior. In this work, a spiking neuron is used to compute a PID control by coding in time the peaks generated by the neuron. The neuron has as synaptic weights the PID gains, and the peak observed in the axon is the coded control signal. The neuron adaptation tries to obtain the necessary weights to generate the peak instant necessary to control the chemical reaction. The simulation results show the possibility of using this kind of neuron for control issues and the possibility of using a spiking neural network to overcome the undershoot obtained due to the inverse response of the chemical reaction. the derivative of the error signal. The synaptic weights can be adapted to minimize the error found between the desired output of the neuron and its current response; this is the backpropagation training method. In this case, these kinds of neurons are known as the second generation of ANN [2].
The neurons of the second-generation work in the following way: some input nodes represent the dendrites of a real neuron, they collect the input information, then the inputs are modified using certain gains that are known are synaptic weights, if these gains are big, they produce the excitation of the neuron, and if they are small or negative, then they produce the inhibition of the neuron. Now, inside the neuron, the input information affected by the weights is added, and if they exceed an internal bias, the neuron is activated; the sum computed inside the neuron is similar to the soma in real neurons. Finally, the output signal is carried out through the axon; this idea is made by evaluating the addition of the information using a nonlinear function [3].
When some neurons are organized in parallel, i.e., there exists only one layer of neurons, the ANN is known as a perceptron, this kind of ANN is evaluated quickly, and its training is also fast, the perceptron is a good proposal for image recognition [4]. When they have connected some perceptrons, the ANN is known as a multilayer perceptron or multilayer ANN. In the control issue, they have used two neuron layers. The first one is known as the hidden layer [5]. The more common activation function for the hidden layer is the hyperbolic tangent because they can be handled by positive and negative values. The second later is the output layer that has as many neurons as degrees of freedom of the dynamic system, and its activation function used to be a linear function.
Recently, Deep Learning (DL) has appeared thanks to the ideas of Hinton and his working team [6]. They affront the problem of adapting a multilayer perceptron if there are used more than three layers because the gradient used in the backpropagation vanishes. DL uses ideas different from the ANN. It uses Boltzmann machines [7], auto-encoders [8], or convolution functions [9] for pre-training. When the system has an acceptable behavior, the new structure is quite similar to a classical ANN (second generation). Now a fine-tuning is made using the well-known backpropagation [10].
The DL can be considered into the fourth industrial revolution, even as the third generation of the ANN, but there exists another kind of artificial neuron. In control, issues are not common to find DL applications. With the ideas and functions proposed by Izhikevich [11], the spiking neurons were easy to compute, and a new paradigm appeared. The idea was to obtain activation signals more similar to the real behavior of biological neurons. Now, some ideas as to building electronic devices capable of computing spiking neural networks have been developed [12]. This paper is shown an application of a PID controller where the gains are computed using spiking neurons. A codification is needed to interpret the spikes in order to obtain a continuous control signal. The classical codifications are in time, taking into consideration when the spike appears [13], or in frequency by averaging the pulses obtained into a time interval.
The system to control is the Van de Vusse reaction. This chemical process presents an inverse response. It is a common situation in chemical and biotechnological processes because the system input appears with a negative sign in all the differential equations that represent the evolution of the concentration in time according to the mathematical model. The Van de Vusse reaction has been treated as a benchmark to propose control strategies for systems with an inverse response or, in its linear version, systems with non-minimum phases [14] [15]. Some control techniques have been used for this chemical reaction as the classical proportional, integral, and derivative (PID) control [15], state feedback regarding the unstable zero [16], internal mode control [17], fuzzy control, and its comparison with type-2 fuzzy control [18], and even ANN by approaching a neural model of the chemical reaction [19]. The spiking neural networks have also been used for control applications. To control a robot, reference [20] was developed an evolving spiking neural network. Reference [21] was designed a spiking PID controller for a UAV. To control nonlinear systems, reference [22] is shown a more general structure using a supervised learning stage with an evolutionary algorithm. In this article, a spiking neuron having as synaptic weights the PID gains are used to control the Van de Vusse chemical reaction. In this way, only one neuron is capable of obtaining the control signal. The synaptic weights are obtained using the spike prop algorithm, a similar technique to the backpropagation. An interesting topic is how to reduce the undershoot obtained in the simulation. One way to avoid this situation is by using an antiwindup term in the control signal. This work presented the spiking PID control application using only one spiking neuron to show how it works and to propose alternatives for enhancing the neuron behavior. For example, in [21], three neuron layers were used to compute different instances in the controller.
The research contribution is to update the synaptic weights by using a model reference in a similar way to the reported in [11], where the control signal is approached after training a spiking neuron with the control signal from a linear PID. In this paper is used the difference between the output of the chemical reaction and the output of the model is a reference to compute the control signal, and then the synaptic weights are updated. One problem is the signal coding because the time resolution used to compute the vector time affects the possible values decoded after the neuron computation. On the other hand, the undershoot shown by the Van de Vusse reaction does not appear in the model reference, and without an anti-windup term, it was not possible to reduce this undesirable effect due to the inverse response.
The rest of the paper is organized as follows: first, in Section 2, the spiking neurons are described. Section 3 shows the Van de Vusse reaction. Section 4 presents the training of the spiking neurons to control the chemical reaction. Section 5 shows the results and discussion. Finally, the conclusions are presented in Section 6.

Spiking Neurons
To describe the behavior of biological neurons, many biophysical interactions are needed, thus in [11] was shown a bifurcation methodology, and now two differential equations are capable of reproducing the signals obtained from real neurons. These signals show some spikes after a certain internal potential is reached. Fig. 1 shows some spikes in shape obtained when an external stimulus makes a rat respond. The differential equations that can describe the spikes obtained are shown by (1) and (2): where ( ) represents the membrane potential of the neuron, ( ) is the potential membrane recovery. These variables are reset when the membrane potential reaches 30mV. This value is approximated the same as biological neurons show. Depending on value, the internal potential is around -65mV, and according to the membrane potential evolution, the threshold potential is around -50mV.
Once the spike is produced, a reset in the neuron is made, as the graphic in Fig. 1 shows; this reset is computed with the auxiliary condition (3): An interesting result is shown in [11], where Izhikevich showed the possible spiking shapes when different values of parameters , , , and are used in the neuron model. Also, a program that simulates 1000 randomly coupled spiking neurons is presented in his paper. Fig.  2 shows a simulation of membrane potential described by (1). It is possible to see the peaks obtained in the simulation. After each peak, a reset is applied, but as the external stimulus ( ) is maintained, the spikes continue. To communicate between neurons, peaks are necessary to simulate the response of a previous neuron. As was aforementioned, there are some techniques to code the information obtained by the peaks given by neurons. In this way, Fig. 3 shows the spiking neuron response when pulses are applied as input. The time when peaks are obtained can be interpreted as the neuron response for situations where more than one peak is generated. Usually, the first peak is only considered. The following step is to find when a peak takes place, and if different neurons are used, each one gives some information regarding the instant of the first pulse it obtained, talking about [11], a simulation or 1000 random connection between spiking neurons is shown in its Fig. 3, this result seems not to be relevant initially, only the fast computing to simulate the peak was important, but when medical researchers saw that graphic, they interpret alpha (10Hz) and gamma (40Hz) rhythms, and when they realized that a random ANN was organized alone, these results were very popular. Fig. 4 shows the simulation of two spiking neurons, and they are presented as the peak instants. Some points as presented are the 1000 points commented in [11].

Time to First Spike Coding
For control issues, for example, to propose spiking PID, it is possible to assign each component to a certain code. The coding consists in to give a time window, where the peak location corresponds to a certain position into the window that corresponds to a scaled output. First, it is necessary to determine some constants, and represent the cotes for the input, now and are cotes for the time window. Finally, to code in time some variables like the error ( ), its derivative named ( ) and its integral respect to time ( ) is possible to use: In order to have pulses when a signal increases, only some kind of spiking signals must be considered. The shape depends on the parameters selected, as was aforementioned. Fig. 2 of [11] shows some of them; in this work are proposed six peaks shapes that are shown in Fig. 5, where also the parameters used to have these responses are presented.
In a similar way, the reference signal can be coded in time ( ), now the synaptic weights can be computed using a time window and a vector that contains the error coding = ( ), the weights are defined by (7) in order to have a smooth function to compute its gradient: where , T and N are parameters to be proposed, can be a new for a new time window. Therefore, the spiking neuron can be seen as a neuron that has three inputs defined into the vector τ. The weights computing were obtained using element by element operations because (7) shows a division and uses an exponential function. These ideas are taken from [23]. Thus a gradient can be computed to update the synaptic weights used in the spiking neuron.
Once the initial weights are obtained, the input vector is obtained using: And the pondered input is = . In (7), the time is evaluated until >= . In that case, the cycle breaks, and is not updated.

Weights Updating and Decoding
The function used to describe the weights is differentiable, now using the gradient is computed the change that must be added to the current weights by using (8), and the updating of the weights is ← + . This is the procedure always that < . Similar to the backpropagation algorithm used in ANN, for SNN, the procedure used to be called spike prop. In case of x exceeds the threshold used, the variation of the weights is made by using: Being the same or a different initial value for a time window, and η is the learning rate. Once the weights were updated. The control signal u(t) can be decoded by proposing a signal interval with the cotes ( ) and using:

Van de Vusse reaction
A benchmark to work with non-minimum-phase systems because they show an inverse response is the Van de Vusse chemical reaction. The process consists of the production of cyclopentenyl B from cyclopentadiene A by using acid-catalyzed electrophilic addition of water [24]. This reaction is described by (13) and (14) where is the cyclopentenyl taken as the process output [23]. The process input is the dilution rate ( ) which is the quotient ( )/ ( ), being ( ) the inlet flow and ( ) the liquid volume inside the reactor, this input ( ) use to be used for control simplification in chemical processes [25].
According to [17], this chemical reaction has the following characteristics:  Input multiplicity, in this way, the Van de Vusse reaction is not controlled affine.
 The reaction has a sign change producing a non-minimum phase behavior.
 Asymmetric response due to nonlinearities in the model.  A delay due to measuring instruments is often not taken into account.
A scheme that represents the Van de Vusse reaction is shown in Fig. 6, where the PID control is obtained by using a spiking neuron, being the error, its derivative, and its integral coded in time, and they represent peaks as Fig 2.1 in [23].
In this kind of system, the inverse response makes the system reacts in the opposite direction to the input at the beginning of the response, this can be seen as a delay in the response, and anti-windup action can be applied to reduce this undesired effect. The control scheme is shown in Fig. 7, where the control gains are updated to have the desired response. Reference [26] were necessary 4ms to adapt the synaptic weights in a leaky integrate and fire spiking model. The application they showed was a haptic interface using quaternion spike neurons.
The spiking control is obtained by computing the PID gains in order to follow a model reference, the spike prop algorithm obtains information from the process output and the model reference, then a change for the synaptic weights is obtained using (8) or (11) according to the threshold value , with these new gains the spiking neuron computes the control signal ( ) using (12), and the signals like the derivative of error or the process output are coded in time to interpret the information into the spiking neuron.

Results and Discussion
The spiking PID computed its gains between samples, it was determined as a limit of iterations to train the neuron in 35 iterations, after the simulation, it was obtained an average of 28.387 iterations to compute the PID gains, and the standard deviation was 4.2698 iterations. Fig. 8 plotted the membrane potential evolution through iterations until converge. The sampling time used was 0.02 min, i.e., 1.2 s, the time used to compute 35 iterations was 645ms with a standard deviation of 4.2 ms, the machine used was a computer with 4GB in RAM and a processor of intel core™ 2.3GHz. However, they were necessary around 28.387 iterations to compute the control gains instead of the limit of 35. The control simulation is shown in Fig. 9, where it is possible to appreciate oscillations at the beginning of the simulation. They are due to the changes obtained in the gains because the spiking neuron tries to obtain a certain PID. As time increase, the gains converge in order to eliminate the steadystate error. The Van de Vusse reaction has an inverse response, and it can be seen that when the oscillations in the simulation are small, the gains are obtained to generate an undesired undershoot. One alternative is to saturate the control gains, the control signal, or even an early stop in the spiking neuron adaptation to avoid the overfitting. In this work, the time window in the peaks coding was used to regulate the undershoot. However, it was necessary to consider if the settling time or the undershoot is the most important characteristic in the simulation. The learning rate also reduces the settling time but increases the initial oscillations. The PID gains are computed each sampling time, they are updated at most 35 times between samples, and their evolution is shown in Fig. 10. Table 1 are shown the initial conditions and parameters used in the simulation.
This work was used five synaptic connections were as inputs to the spiking neuron. They are the reference signal, the error as the difference between the process output and the reference signal, their derivative and integral, and a measure about the undershoot in order to adapt the time window. In [17], an interesting result about the control of the Van de Vusse reaction is shown in Fig. 6, where was used a nonlinear control strategy, and the different results were obtained by varying the parameter 1 in the model equations (13) and (14).

Conclusions
Spiking neurons are an interesting topic because they simulate the real behavior depicted by biological neurons. To obtain peaks by stimulating the neurons was easy using the model shown in [11]; now, each response obtained by a real neuron is possible. An important issue is to decode the peaks given as neuron responses. This work was used time coding to interpret the signal obtained from the neuron.  In a similar way to the backpropagation algorithm used to train ANN, some algorithms use similar ideas as the spike prop and the quick prop. In this paper, the PID gains are proposed by reducing the output error inside a spiking neuron. To adapt the PID gains, which are the synaptic weights of the spiking neuron, a reference model was used to change the closed-loop behavior. A small learning rate can reduce the initial oscillation in the simulation, but sometimes are necessary more iteration for the weight adaptation. Thus the learning rate was selected in order to require less than 35 iterations.
As control application was used, the Van de Vusse reaction, a benchmark used to work with systems with an inverse response due to its non-minimum-phase characteristic. The results are possible to see the control simulation. The main drawbacks are the oscillation obtained until the convergence of the parameters is achieved and the undershooting due to the inverse response must be reduced by aggregating another control technique, for example an anti-windup term.