classification of gemstones

cnn with explainability for fun

CODE

For this project, I wanted to classify gemstones using deep convolutional neural networks and then use an explainability method to be able to identify which features influence the prediction the most.

I used the gemstones dataset from Daria Chemkaeva. It includes 2856 images belonging to 87 classes.

Since the shape of the gemstone is not really important, but the color and the patterns of the gem, I performed a data augmentation step.

Then I built a convolutional neural network with residual blocks and batch normalization. I trained the model for 137 epochs, due to the early stopping. The accuracy on the training set was around 80% and the accuracy on the validation around 70%.

And of course, when implementing the VarGrad (VG) explainability method, we can see nothing in the score maps. This is only confirming what we already know, that the categorisation of gemstones is done through the color. And therefore, the performance may also not be so high, because we have gemstones with very similar colors and patterns.

However, if we use a different dataset, we may obtain better results. For example, for the classification of buildings according to their architectural style or more complex things, such as predicting the transcription rates from multiplexed protein maps .

Score map and original photo when classificating of buildings according to their architectural style, using the Architectural styles dataset.

Thanks to Andres Becker for helping me to build this project!