Handwritten Signature Recognition is an important behavioral biometric which is used for numerous identification and authentication applications. There are two fundamental methods of signature recognition, on-line or off-line. On-line recognition is a dynamic form, which uses parameters like writing pace, change in stylus direction and number of pen ups and pen downs during the writing of the signature. Off-line signature recognition is a static form where a signature is handled as an image and the author of the signature is predicted based on the features of the signature. The current method of Off-line Signature Recognition predominantly employs template matching, where a test image is compared with multiple specimen images to speculate the author of the signature. This takes up a lot of memory and has a higher time complexity. This paper proposes a method of off-line signature recognition using Convolution Neural Network. The purpose of this paper is to obtain high accuracy multi-class classification with a few training signature samples.