Image Classifier (ML)

Image Classifier is the extension of the ML Environment that deals with the classification of the images into different classes.

For example, let’s say you want to construct a model to judge if a person is wearing a mask correctly or not, or if the person’s wearing one at all.

This is the case of image classification where you want the machine to label the images into one of the classes.

Tutorial on using Image Classifier in Block Coding

Tutorial on using Image Classifier in Python Coding

Opening Image Classifier Workflow

Alert: The Machine Learning Environment for model creation is available in the only desktop version of PictoBlox for Windows, macOS, or Linux. It is not available in Web, Android, and iOS versions.

Follow the steps below:

1. Open PictoBlox and create a new file.
2. Select the coding environment as appropriate Coding Environment.
   
3. Select the “Open ML Environment” option under the “Files” tab to access the ML Environment.
   
4. You’ll be greeted with the following screen.
   Click on “Create New Project“.
5. A window will open. Type in a project name of your choice and select the “Image Classifier” extension. Click the “Create Project” button to open the Image Classifier window.
6. You shall see the Image Classifier workflow with two classes already made for you. Your environment is all set. Now it’s time to upload the data.

Class in Image Classifier

Class is the category in which the Machine Learning model classifies the images. Similar images are put in one class.

There are 2 things that you have to provide in a class:

1. Class Name: It’s the name to which the class will be referred as.
2. Image Data: This data can either be taken from the webcam or by uploading from local storage.

Note: You can add more classes to the projects using the Add Class button.

Adding Data to Class

You can perform the following operations to manipulate the data into a class.

1. Naming the Class: You can rename the class by clicking on the edit button.
2. Adding Data to the Class: You can add the data using the Webcam or by Uploading the files from the local folder.
   

   Note: You can edit the capture setting in the camera with the following. Hold to Record allows you to capture images till the time button is pressed. Whereas when it is off you can set the start delay and duration of the sample collection.

   

   If you want to change your camera feed, you can do it from the webcam selector in the top right corner.
   

3. Deleting individual images:
4. Delete all images:
5. Enable or Disable Class: This option tells the model whether to consider the current class for the ML model or not. If disabled, the class will not appear in the ML model trained.
6. Delete Class: This option deletes the full class.

Note: You must add at least 20 samples to each of your classes for your model to train. More samples will lead to better results.

Training the Model

After data is added, it’s fit to be used in model training. In order to do this, we have to train the model. By training the model, we extract meaningful information from the images, and that in turn updates the weights. Once these weights are saved, we can use our model to make predictions on data previously unseen.

However, before training the model, there are a few hyperparameters that you should be aware of. Click on the “Advanced” tab to view them.

Note: These hyperparameters can affect the accuracy of your model to a great extent. Experiment with them to find what works best for your data.

There are three hyperparameters you can play along with here:

1. Epochs– The total number of times your data will be fed through the training model. Therefore, in 10 epochs, the dataset will be fed through the training model 10 times. Increasing the number of epochs can often lead to better performance.
2. Batch Size– The size of the set of samples that will be used in one step. For example, if you have 160 data samples in your dataset, and you have a batch size of 16, each epoch will be completed in 160/16=10 steps. You’ll rarely need to alter this hyperparameter.
3. Learning Rate– It dictates the speed at which your model updates the weights after iterating through a step. Even small changes in this parameter can have a huge impact on the model performance. The usual range lies between 0.001 and 0.0001.

Note: Hover your mouse over the question mark next to the hyperparameters to see their description.

It’s a good idea to train a numeric classification model for a high number of epochs. The model can be trained in both JavaScript and Python. In order to choose between the two, click on the switch on top of the Training panel.

Alert: Dependencies must be downloaded to train the model in Python, JavaScript will be chosen by default.

The accuracy of the model should increase over time. The x-axis of the graph shows the epochs, and the y-axis represents the accuracy at the corresponding epoch. Remember, the higher the reading in the accuracy graph, the better the model. The x-axis of the graph shows the epochs, and the y-axis represents the corresponding accuracy. The range of the accuracy is 0 to 1.

Testing the Model

To test the model, simply enter the input values in the “Testing” panel and click on the “Predict” button.

The model will return the probability of the input belonging to the classes.

Export in Block Coding

Click on the “Export Model” button on the top right of the Testing box, and PictoBlox will load your model into the Block Coding Environment if you have opened the ML Environment in the Block Coding.

Export in Python Coding

Alert: For the model to work in Python Coding Environment the model is need to be trained in Python.

Click on the “Export Model” button on the top right of the Testing box, and PictoBlox will load your model into the Python Coding Environment if you have opened the ML Environment in Python Coding.

The following code appears in the Python Editor of the selected sprite.

####################imports####################
# Do not change

import cv2
import numpy as np
import tensorflow as tf

# Do not change
####################imports####################

#Following are the model and video capture configurations
# Do not change

model = tf.keras.models.load_model('saved_model.h5',
                                   custom_objects=None,
                                   compile=True,
                                   options=None)

cap = cv2.VideoCapture(0)  # Using device's camera to capture video
text_color = (206, 235, 135)
org = (50, 50)
font = cv2.FONT_HERSHEY_SIMPLEX
fontScale = 1
thickness = 3

class_list = ['Mask Off', 'Mask On', 'Mask Wrong']  # List of all the classes

# Do not change
###############################################

#This is the while loop block, computations happen here

while True:
  ret, image_np = cap.read()  # Reading the captured images
  image_np = cv2.flip(image_np, 1)
  image_resized = cv2.resize(image_np, (224, 224))
  img_array = tf.expand_dims(image_resized,
                             0)  # Expanding the image array dimensions
  predict = model.predict(
      img_array)  # Making an initial prediction using the model
  predict_index = np.argmax(predict[0],
                            axis=0)  # Generating index out of the prediction
  predicted_class = class_list[
      predict_index]  # Tallying the index with class list

  image_np = cv2.putText(
      image_np, "Image Classification Output: " + str(predicted_class), org,
      font, fontScale, text_color, thickness, cv2.LINE_AA)

  cv2.imshow("Image Classification Window",
             image_np)  # Displaying the classification window

  if cv2.waitKey(25) & 0xFF == ord(
      'q'):  # Press 'q' to close the classification window
    break

cap.release()  # Stops taking video input
cv2.destroyAllWindows()  # Closes input window

Note: You can edit the code to add custom code according to your requirement.