Journal Publications

● Steven Spielberg P, Aditya Tulsyan, Nathan P Lawrence, Philip D Loewen and R Bhushan Gopaluni, Toward self‐driving processes: A deep reinforcement learning approach to control, AIChE Journal, 2019 Link

● M Wallace, Steven Spielberg P and P Mhaskar, Offset-Free Model Predictive Control (MPC) with Explicit Performance Specification, I&ECR, 2016. Link

● Steven Spielberg P, Aditya Tulsyan, Nathan P Lawrence, Philip D Loewen and R Bhushan Gopaluni, Deep Reinforcement Learning for Process Control: A Primer for Beginners, arXiv Print, 2020 Link PDF

Conference Publications

● Steven Spielberg P, R. B. Gopaluni and P. D Lowen, A Deep Learning Architecture for Predictive Control, Advanced Control of Chemical Processes, Liaoning, China, 2018 Link PDF

● Steven Spielberg P, Bhushan Gopaluni and Philip D. Loewen, Deep Reinforcement Learning Approaches for Process Control, Advanced Control of Industrial Processes, Taiwan, 2017. Link PDF

Software

Conference Presentations

● Steven Spielberg P, R. B. Gopaluni and Philip D. Loewen, Deep Reinforcement Learning Approaches for Process Control, AICHE Annual Meeting, San Fransisco, 2016. Link

● Steven Spielberg P, R. B. Gopaluni and Philip D. Loewen, Deep Reinforcement Learning Approaches for Process Control, Advanced Control of Industrial Processes, Taiwan, 2017.

● Steven Spielberg P, R. B. Gopaluni, A Deep Learning Architecture for Predictive Control, CSCHE, Alberta, Canada, 2017 Slides

● Steven Spielberg P, R. B. Gopaluni and P. D Lowen, A Deep Learning Architecture for Predictive Control, Advanced Control of Chemical Processes, Liaoning, China, 2018 [Accepted]

Talks

● Gaussian Copula Models, UBC Machine Learning Reading Group, Summer, 2016. Link Slides

● Recurrent Neural Networks, UBC Machine Learning Reading Group, Fall, 2016. Link Slides

● Deep Reinforcement Learning Approaches for Process Control (2016 - 2017)

We extended the current success of deep learning and reinforcement learning to process control problems. We have shown that if reward hypothesis is properly formulated, then reinforcement learning agent can be used for industrial process control. We presented an actor-critic based algorithm and tested it on Single Input Single Output systems (SISO) and Non-linear Multi Input Multi Output Systems (MIMO) under various scenarios.

● Control of Dynamic Systems Using LSTM supported Neural Network (Spring 2016)

We designed a neural network architecture combining Long Short Term Memory (LSTM) and Neural Network (NN) called LSTM Supported NN (LSTMSNN). This specific architecture (LSTMSNN) accounts for past information and present behaviour of the system and was used to learn the complex behaviour of MPC. MPC is used to generate data for training the network. This learned network acquired a policy that maps system ouptut to control action without having to estimate the state and its fast during test time. We evaluated our trained model on varying target outputs, various initial conditions and compared it with other ordinary NN and LSTM.

Code PDF

● Reinforcement Learning with Functional Approximation Using a Neural Network (Fall 2015)

Trained a robot in robocode platform (written in java) using Q-learning and Sarsa. Intially, Q-learning was implemented with state-action pairs and corresponding Q-values stored in a look up table (LUT). This involved storing huge amount of parameters(15360) and quatization of states. Later, the state-action pairs and Q-values were learnt online using 19 hidden units neural network. This allowed more precision in states and fewer parameters (134). The trained robot had 91% winning rate.

Code PDF

● Data Visualization of World Crude Oil Supply (Oct-Nov 2015)

Collected the world crude oil supply from onshore and offshore data from Peace Research Institute Oslo (PRIO). The data containing the city names were converted to coordiantes using webparsing. Used Webgl globe to perform a 3D visualization. Scripts used: node.js, three.js.

Link

● Fuzzy Model Clustering for Multi-Modal Learning (June-Aug 2015)

Company: Gyan Data Pvt. Ltd., Chennai, India.

Identified the parameters of multiple linear models present in the data without having prior information about number of models and model parameters. The algorithm uses Fuzzy C Means Clustering Algorithm as a basis. The distance measure in Fuzzy C Means is the Euclidean distance (distance between Cluster Centres and data Points) whereas the Fuzzy Model Clustering (FMC) Algorithm uses prediction error (difference between output and predicted output) as the distance measure to identify the parameter in the model. The algorithm was coded in Python and tested on foundry data. The parameters were used to identify the features that played significant role in casting.

Code

● Model Prective Control of an Fluid Catalytic Cracking (FCC) Unit (Dec-14 Jan-2015)

Project Mentor:Dr. Raghunathan Rengaswamy, Professor, Department of Chemical Engg., Indian Institute of Technology, Madras (IITM)

The non linear model of FCC unit was used to develop an appropriate state space Linear Time Invariant Model. Subspace identification (N4SID-Matlab) method was used to develop the state space using Pseudo Random Binary Signal and IRS input sequences. Performed MPC on the obtained state space model. The states were estimated using Kalman Filter. Coded the MPC from scratch and setpoints were tracked precisely when compared to Proportional Integral Derivative Controller.

● Model Predictive Control (MPC) Tuning Method (May-July, 2014)

Project Mentor: Dr. Prashant Mhaskar, Associate Professor, McMaster Advanced Control Consortium (MACC) , McMaster University, Canada
Program: Mitacs Globalink

We designed a two-tiered optimization approach to Model Predictive Control subject to input-constraints and plant-model mismatch. The first tier computes the best response of a certain desired form (fastest first-order response/fastest second-order response with minimum overshoot ratio). The second tier computes the control action to achieve the prescribed closed-loop response. Formulated MPC was found to be less oscillatory and attained faster steady-state convergence compared to traditional offset free MPC.

● Mathematical Modeling of Solar Still (June-July 2013, Nov-Dec 2013)

Project Mentor: Dr. Pratap S Bapat, Scale up and Processing Unit, Central Salt and Marine Chemicals Research Institute (CSMCRI), Gujarat, India
Program: Science Academies Fellowship

Developed mathematical model for solar still which involves ordinary and partial differential equations. The equations were solved to study the effect of distance between water and condensate (glass) surface on the amount of water condensed. Wrote a C-program to study the same for various parameters in the system. It was found that solar still with small gap between water and glass surface had higher rate of condensation compared to one with larger gap. The model was developed with the inclusion of all modes of heat transfer and disturbances caused by wind.