View article

In this work, deep transfer learning is proposed for recognition of sign sequence in sentences continuously signed in the Indian sign language using sufficient labelled data of isolated signs and limited amount of labelled sentence data. The data is collected using multiple six degree-of-freedom inertial measurement units (IMUs) on both hands of the signer. The proposed deep learning model consists of convolutional neural network (CNN), two bidirectional long short-term memory (Bi-LSTM) layers and connectionist temporal classification (CTC) to enable end-to-end sentence recognition without requiring the knowledge of sign boundaries. Initially, the network is trained on isolated signs data. Based on the hypothesis that generic features learned from isolated signs will enhance the classification of continuous sentence sign data, a novel transfer learning framework is proposed, wherein last few layers of the …