[PictoBloxExtension]

Image Classifier (ML)

Image Classifier icon
Extension Description
Create ML models to classify images into different classes.

Introduction

Image Classifier is the extension of the ML Environment that deals with the classification of the images into different classes.

For example, let’s say you want to construct a model to judge if a person is wearing a mask correctly or not, or if the person’s wearing one at all.

This is the case of image classification where you want the machine to label the images into one of the classes.

Tutorial on using Image Classifier in Block Coding

Tutorial on using Image Classifier in Python Coding

Opening Image Classifier Workflow

Alert: The Machine Learning Environment for model creation is available in the only desktop version of PictoBlox for Windows, macOS, or Linux. It is not available in Web, Android, and iOS versions.

Follow the steps below:

  1. Open PictoBlox and create a new file.
  2. Select the coding environment as appropriate Coding Environment.
  3. Select the “Open ML Environment” option under the “Files” tab to access the ML Environment.
  4. You’ll be greeted with the following screen.
    Click on “Create New Project“.
  5. A window will open. Type in a project name of your choice and select the “Image Classifier” extension. Click the “Create Project” button to open the Image Classifier window.
  6. You shall see the Image Classifier workflow with two classes already made for you. Your environment is all set. Now it’s time to upload the data.

Class in Image Classifier

Class is the category in which the Machine Learning model classifies the images. Similar images are put in one class.

There are 2 things that you have to provide in a class:

  1. Class Name: It’s the name to which the class will be referred as.
  2. Image Data: This data can either be taken from the webcam or by uploading from local storage.

Note: You can add more classes to the projects using the Add Class button.

Adding Data to Class

You can perform the following operations to manipulate the data into a class.

  1. Naming the Class: You can rename the class by clicking on the edit button.
  2. Adding Data to the Class: You can add the data using the Webcam or by Uploading the files from the local folder.
    Note: You can edit the capture setting in the camera with the following. Hold to Record allows you to capture images till the time button is pressed. Whereas when it is off you can set the start delay and duration of the sample collection.

    If you want to change your camera feed, you can do it from the webcam selector in the top right corner.

  3. Deleting individual images:
  4. Delete all images:
  5. Enable or Disable Class: This option tells the model whether to consider the current class for the ML model or not. If disabled, the class will not appear in the ML model trained.
  6. Delete Class: This option deletes the full class.
Note: You must add at least 20 samples to each of your classes for your model to train. More samples will lead to better results.

Training the Model

After data is added, it’s fit to be used in model training. In order to do this, we have to train the model. By training the model, we extract meaningful information from the images, and that in turn updates the weights. Once these weights are saved, we can use our model to make predictions on data previously unseen.

However, before training the model, there are a few hyperparameters that you should be aware of. Click on the “Advanced” tab to view them.

Note: These hyperparameters can affect the accuracy of your model to a great extent. Experiment with them to find what works best for your data.

There are three hyperparameters you can play along with here:

  1. Epochs– The total number of times your data will be fed through the training model. Therefore, in 10 epochs, the dataset will be fed through the training model 10 times. Increasing the number of epochs can often lead to better performance.
  2. Batch Size– The size of the set of samples that will be used in one step. For example, if you have 160 data samples in your dataset, and you have a batch size of 16, each epoch will be completed in 160/16=10 steps. You’ll rarely need to alter this hyperparameter.
  3. Learning Rate– It dictates the speed at which your model updates the weights after iterating through a step. Even small changes in this parameter can have a huge impact on the model performance. The usual range lies between 0.001 and 0.0001.
Note: Hover your mouse over the question mark next to the hyperparameters to see their description.

It’s a good idea to train a numeric classification model for a high number of epochs. The model can be trained in both JavaScript and Python. In order to choose between the two, click on the switch on top of the Training panel.

Alert: Dependencies must be downloaded to train the model in Python, JavaScript will be chosen by default.

The accuracy of the model should increase over time. The x-axis of the graph shows the epochs, and the y-axis represents the accuracy at the corresponding epoch. Remember, the higher the reading in the accuracy graph, the better the model. The x-axis of the graph shows the epochs, and the y-axis represents the corresponding accuracy. The range of the accuracy is 0 to 1.

Testing the Model

To test the model, simply enter the input values in the “Testing” panel and click on the “Predict” button.

The model will return the probability of the input belonging to the classes.

Export in Block Coding

Click on the “Export Model” button on the top right of the Testing box, and PictoBlox will load your model into the Block Coding Environment if you have opened the ML Environment in the Block Coding.

Export in Python Coding

Alert: For the model to work in Python Coding Environment the model is need to be trained in Python.

Click on the “Export Model” button on the top right of the Testing box, and PictoBlox will load your model into the Python Coding Environment if you have opened the ML Environment in Python Coding.

The following code appears in the Python Editor of the selected sprite.

####################imports####################
# Do not change

import cv2
import numpy as np
import tensorflow as tf

# Do not change
####################imports####################

#Following are the model and video capture configurations
# Do not change

model = tf.keras.models.load_model('saved_model.h5',
                                   custom_objects=None,
                                   compile=True,
                                   options=None)

cap = cv2.VideoCapture(0)  # Using device's camera to capture video
text_color = (206, 235, 135)
org = (50, 50)
font = cv2.FONT_HERSHEY_SIMPLEX
fontScale = 1
thickness = 3

class_list = ['Mask Off', 'Mask On', 'Mask Wrong']  # List of all the classes

# Do not change
###############################################

#This is the while loop block, computations happen here

while True:
  ret, image_np = cap.read()  # Reading the captured images
  image_np = cv2.flip(image_np, 1)
  image_resized = cv2.resize(image_np, (224, 224))
  img_array = tf.expand_dims(image_resized,
                             0)  # Expanding the image array dimensions
  predict = model.predict(
      img_array)  # Making an initial prediction using the model
  predict_index = np.argmax(predict[0],
                            axis=0)  # Generating index out of the prediction
  predicted_class = class_list[
      predict_index]  # Tallying the index with class list

  image_np = cv2.putText(
      image_np, "Image Classification Output: " + str(predicted_class), org,
      font, fontScale, text_color, thickness, cv2.LINE_AA)

  cv2.imshow("Image Classification Window",
             image_np)  # Displaying the classification window

  if cv2.waitKey(25) & 0xFF == ord(
      'q'):  # Press 'q' to close the classification window
    break

cap.release()  # Stops taking video input
cv2.destroyAllWindows()  # Closes input window
Note: You can edit the code to add custom code according to your requirement.
Read More

PictoBlox Blocks

Turn () video on stage with () % transparency block controls the control the camera feed on the stage.
This block helps move an object step by step in a given direction within a set time
This block facilitates seamless transitions to your preferred scenes. For instance, if you’re currently viewing Scene 1 and wish to switch to Scene 2, simply select Scene 2 from the dropdown menu to enact the transition.
This block is designed to refresh or reload the current scene.
This block code will give an option to control the object as player control like third person or first person. You can then use arrow keys to make your object move in respective direction.
This block of code applies velocity to the player. 
All articles loaded
No more articles to load

Block Coding Examples

All articles loaded
No more articles to load

Python Functions

The function creates an object to connect Quarky with Wi-Fi.
Syntax: wifi()
The function reads the analog value of the sensors connected to the specified pin. The function returns the int value between 0 to 4096.
Syntax: readanaloginput(pin = “A1”)
The function is used to control the state of the camera.
Syntax: video(video_state = “on”, transparency = 1)
The function moves its sprite forward the specified amount of steps in the direction it is facing. A step is equal to one-pixel length.
Syntax: move(steps = 10)
The function gives its sprite a speech bubble with the specified text — the speech bubble stays until another speech or thought block is activated, or the stop sign is pressed.
Syntax: say(message = “Hello!”, time = 0)
The function will play the specified sound, pausing its script until the sound has finished playing.
Syntax: playuntildone(sound_name = “Grunt”)
The function checks if its sprite is touching the mouse-pointer, edge, or another sprite. If the sprite is touching the selected object, the block returns true; if it is not, it returns false.
Syntax: istouching(object_name = “_edge_”)
The function is used when the sprite needs to produce a bitmap image of itself which is stamped onto the stage. (Because it is merely a picture of the sprite and not a sprite itself, it cannot be programmed). The function will not draw over sprites. 
Syntax: stamp()
The function sets the specified RGB LED of Quarky (specified with X and Y position of the LED) to the specified RGB color and brightness value.
Syntax: setled(x_position = 1, y_position = 1, color = [0, 0 , 0], brightness = 20)
The function returns the state of the specified push button. If the button is pressed it returns True or else False.
Syntax: readpushbutton(button = “L”)
The function moves the Quarky robot in the specified direction. The direction can be “FORWARD”, “BACKWARD”, “LEFT”, and “RIGHT”.
Syntax: runrobot(direction = “FORWARD”, speed = 100)
The function plays the specified audio on the Quarky speaker. The function does not have any callbacks, so other functions can be executed while this function is running.
Syntax: playsound(audio = “QuarkyIntro”)
This function helps turn the video on/off on the stage with a defined level of transparency.
Syntax: video(video_state = “on”, transparency = 1)
This function helps turn the video on/off on the stage with a defined level of transparency.
Syntax: video(video_state = “on”, transparency = 1)
The function sets a loudness filter threshold to remove the background noise from the audio file which is being analyzed.
Syntax: setthreshold(loudness = 30)
The function initializes the pick and place robot with the specified orientation.
Syntax: initialisepickplace(orientation = “HORIZONTAL”)
The function loads a model saved via model.save().
Syntax: tf.keras.models.load_model(filepath = ‘saved_model.h5’, custom_objects = None, compile = True, options = None)
The function initializes the quadruped robot object in Python and maps the 8 servos to the specified pins.
Syntax: Quadruped(Front Right Hip = 4, Front Left Hip = 1, Front Right Leg = 8, Front Left Leg = 5, Back Right Hip = 3, Back Left Hip = 2, Back Right Leg = 7, Back Left Leg = 6)
The function set the state of the relay connected to the selected pin to High or Low. A high state means that the pin will have 3.3V and for Low, the pin will be 0 V.
Syntax: setrelay(state = 1, pin = “D3”)
The function connects the Quarky or ESP32 to the specified Wi-Fi and password. The block is only available in the Upload Mode when the code is uploaded to Quarky.
Syntax: wifi.connecttowifi(WIFI = “Wi-Fi Name”, PASSWORD = “password”)
The function reports if the Wi-Fi is connected to the Quarky or ESP32 or not. This block is only available in Upload Mode.
Syntax: wifi.iswificonnected()
The function initializes the Mars Rover object in Python and maps the 5 servos to the specified pins.
Syntax: MarsRover(Head = 4, Front Left = 1, Front Right = 7, Back Left = 2, Back Right = 6)
The function initializes the humanoid robot object in Python and maps the 6 servos to the specified pins.
Syntax: Humanoid(Right Hip = 7, Left Hip = 2, Right Foot = 6, Left Foot = 3, Right Hand = 8, Left Hand = 1)
The Quarky Mecanum Robot Drive Motors are initialized by the function, which assigns each motor to a specific port. This allows the robot to be programmed to move the motors in the desired direction.
Syntax: Mecanum(Front Left = 1, Back Left = 2, Back Right = 7, Front Right = 8)
The function initializes the Expansion Board of Quarky for use. Without initialization, the board will not respond to any other functions.
Syntax: initexpansion()
The function returns the arc cosine of a number.
Syntax: math.acos(x)
This function performs the calibration process for a robotic arm by taking in the error angles of each of its three servos (link 1, link 2, and base). It then stores the offset angle for each servo to the memory of the Quarky, ensuring that the robotic arm is functioning correctly at all times.
Syntax: roboticArm.calibrate(Link1 Offset = 0, Link2 Offset = 0, Base Offset = 0)
This function ensures proper sensor alignment for precise line-following control.
Syntax: AdvanceLineFollowing(ir_num = 2)
The function sets the digital state of the specified pin to HIGH or LOW / 0V or 3.3V.
Syntax: setdigitaloutput(pin, state)
The function enables the automatic display of the box on face detection on the stage.
Syntax: enablebox()
All articles loaded
No more articles to load

Python Coding Examples

All articles loaded
No more articles to load
Table of Contents