Signature Verification Using Unsupervised Learning

Signature Verification is a task to identify whether the given signature is matched with an authentic signature or not. Multiple industries such as Finance, Insurance, Forensic and many others come around in a need for signature verification on an everyday basis which requires a fairly good amount of understanding and a focus to perform it manually by an expert. The process involves checking whether the questioned signature is similar to the signatures recorded under observations or collected from verified documents of the same person.

Computer Vision along with Artificial Intelligence capabilities can be used to build a system to automate the task at a certain level that can match the questioned signature with verified signatures and come up with confidence score/ similarity score which then can be sent for the manual inspection.

In this blog, we have illustrated a system and a case-study which has been designed and implemented for signature verification that uses computer vision and deep learning technologies.

We have built a system which contains three modules:

1. Authorized signatures
2. Question signatures
3. Results

User can upload authorized signatures which will be used to identify the dynamic threshold value for similarity that refers to the possible variations that could be present in multiple signatures from a single person. The system then performs signature similarity between questioned signature and authorized signatures to label whether the questioned signature is matched or not based on similarity score and the calculated threshold value.

But, how does it works !!

We have used an unsupervised approach that uses Transfer Learning to extract features from the signature/ image and computes feature similarity. The entire implementation is performed in Python language using the below tools:

1. Scipy: for vector similarity
2. OpenCV: for image processing operations
3. TensorFlow: for feature extraction

The architecture which has been followed for the Signature Verification task uses VGG16 CVV architecture trained on the Imagenet dataset, and the bottleneck features have been extracted from the last pooling layer of the VGG16 network.

For dynamic threshold value calculation (training module), it takes authorized signature images, extracts bottleneck features, computes cosine similarity amongst all the possible pairs of signatures within the authorized signature corpus of the same person, and calculates the possible variations between pair of signature features that belong to the same person. It then considers the median value of variations from it.

When any new questioned images come (inference module), the system extracts bottleneck features using VGG16 pre-trained model, calculates the cosine similarity between questioned signature features and all other authorized signature features, calculate average similarity and if it is greater than or equals to the threshold value then it labels it as MATCHED (positive sample) else flag it as NOT MATCHED (negative sample).

Case Study

We have tested the solution on open-source data of signature verification having signatures of different 115 unique people, each with 25+ genuine, 30+ simple and skilled forged signatures.

Below is the case-study results that we received from the system on one person’s signatures:

Authorized signatures:

We have considered 25 authorized signatures for threshold value calculation. Below are sample authorized signatures.

Threshold value: 75.5%

Questioned signatures results:

We have tested the model on more than 65 questioned images contains genuine signatures, simple forged and skilled forged signatures. Below are the correctly predicted sample questioned signatures and it’s predictions.

From the analysis, a given skilled forged signature looks much similar to the authorized signature, even though the average similarity for the same is less than the threshold value hence, it is flagged as NOT MATCHED by the model.

We have validated the solution over 100+ test cases and the precision of the model on the validation set marked 99%. However, few of the genuine signatures were also flagged as NOT MATCHED in some of the scenarios.

Summary

The Signature Verification task is crucial to handle automatically as it requires expertise to perform even manually. Here, we have presented the unsupervised approach that computes the similarity of questioned signature with already verified signatures that can help verification experts to prioritize the signatures to be verified from the bunch of signatures. This can improve the efficiency of the manual verification task.

Advanced deep learning algorithms and artificial intelligence techniques can be used to perform supervised learning for signature verification, although the quantity of labeled dataset and data privacy could be the bottleneck for the same.

In addition to signature verification, unsupervised image similarity can be be used for multiple business use-cases in various industries such as product recommendations for e-commerce, face recognition, and so on using different advanced neural networks and computer vision techniques and algorithms.

Reach out to us at info@intellica.ai to know-how and where image similarity can fit into your business to make it more efficient.