[PictoBloxExtension]

Object Detection

Extension Description

AI to identify objects from images. It also tells the location and size of the objects identified.

Available in: Block Coding, Python Coding
Mode: Stage Mode
WiFi Required: No
Compatible Hardware in Block Coding: evive, Quarky, Arduino Uno, Arduino Mega, Arduino Nano, ESP32, T-Watch, Boffin, micro:bit, TECbits, LEGO EV3, LEGO Boost, LEGO WeDo 2.0, Go DFA, None
Compatible Hardware in Python: Quarky, None
Object Declaration in Python: od = ObjectDetection()
Extension Catergory: Artificial Intelligence

Introduction

Object detection is used to locate and identify multiple objects in digital photographs. It is a computer vision technique that helps to detect objects as well as classify them. The object class may appear once or several times in the image. For example, in the following image, object detection assists us to locate the objects and classify them accordingly to the known set of objects.

Object Detection

One of the applications of Object detection includes Self-driving vehicles which detect objects in real-time and act accordingly.

The object detection extension in PictoBlox allows you to detect the following 90 objects:

ID	OBJECT (PAPER)	SUPER CATEGORY
1	person	person
2	bicycle	vehicle
3	car	vehicle
4	motorcycle	vehicle
5	airplane	vehicle
6	bus	vehicle
7	train	vehicle
8	truck	vehicle
9	boat	vehicle
10	traffic light	outdoor
11	fire hydrant	outdoor
12	street sign	outdoor
13	stop sign	outdoor
14	parking meter	outdoor
15	bench	outdoor
16	bird	animal
17	cat	animal
18	dog	animal
19	horse	animal
20	sheep	animal
21	cow	animal
22	elephant	animal
23	bear	animal
24	zebra	animal
25	giraffe	animal
26	hat	accessory
27	backpack	accessory
28	umbrella	accessory
29	shoe	accessory
30	eye glasses	accessory
31	handbag	accessory
32	tie	accessory
33	suitcase	accessory
34	frisbee	sports
35	skis	sports
36	snowboard	sports
37	sports ball	sports
38	kite	sports
39	baseball bat	sports
40	baseball glove	sports
41	skateboard	sports
42	surfboard	sports
43	tennis racket	sports
44	bottle	kitchen
45	plate	kitchen
46	wine glass	kitchen
47	cup	kitchen
48	fork	kitchen
49	knife	kitchen
50	spoon	kitchen
51	bowl	kitchen
52	banana	food
53	apple	food
54	sandwich	food
55	orange	food
56	broccoli	food
57	carrot	food
58	hot dog	food
59	pizza	food
60	donut	food
61	cake	food
62	chair	furniture
63	couch	furniture
64	potted plant	furniture
65	bed	furniture
66	mirror	furniture
67	dining table	furniture
68	window	furniture
69	desk	furniture
70	toilet	furniture
71	door	furniture
72	tv	electronic
73	laptop	electronic
74	mouse	electronic
75	remote	electronic
76	keyboard	electronic
77	cell phone	electronic
78	microwave	appliance
79	oven	appliance
80	toaster	appliance
81	sink	appliance
82	refrigerator	appliance
83	blender	appliance
84	book	indoor
85	clock	indoor
86	vase	indoor
87	scissors	indoor
88	teddy bear	indoor
89	hair drier	indoor
90	toothbrush	indoor
91	hair brush	indoor

Accessing Object Detection in Block Coding

Following is the process to add Object Detection capability to the PictoBlox Project.

Open PictoBlox and create a new file.
Select the coding environment as appropriate Coding Environment.
Next, click on the Add Extension button and add the Object Detection extension.
The object detection models will be downloaded, which may take a few seconds to load.
Once downloaded you will get a message saying “Model loaded successfully”.
You can find the object detection blocks available in the project.

Accessing Object Detection in Python Coding

Following is the process to add Object Detection capability to the PictoBlox Project.

Open PictoBlox and create a new file.
Select the coding environment as Block Coding.
Next, click on the Add Modules/Libraries button and add the Face Detection extension.
To access the library functions, you have to add the object declaration.
```
od = ObjectDetection()
```

PictoBlox Blocks

Turn () video on stage with () % transparency block controls the control the camera feed on the stage.

This block helps move an object step by step in a given direction within a set time

This block facilitates seamless transitions to your preferred scenes. For instance, if you’re currently viewing Scene 1 and wish to switch to Scene 2, simply select Scene 2 from the dropdown menu to enact the transition.

This block is designed to refresh or reload the current scene.

This block code will give an option to control the object as player control like third person or first person. You can then use arrow keys to make your object move in respective direction.

This block of code applies velocity to the player.

All articles loaded

No more articles to load

Block Coding Examples

All articles loaded

No more articles to load

Python Functions

wifi()

The function creates an object to connect Quarky with Wi-Fi.

Syntax: wifi()

readanaloginput()

The function reads the analog value of the sensors connected to the specified pin. The function returns the int value between 0 to 4096.

Syntax: readanaloginput(pin = “A1”)

video()

The function is used to control the state of the camera.

Syntax: video(video_state = “on”, transparency = 1)

move()

The function moves its sprite forward the specified amount of steps in the direction it is facing. A step is equal to one-pixel length.

Syntax: move(steps = 10)

say()

The function gives its sprite a speech bubble with the specified text — the speech bubble stays until another speech or thought block is activated, or the stop sign is pressed.

Syntax: say(message = “Hello!”, time = 0)

playuntildone()

The function will play the specified sound, pausing its script until the sound has finished playing.

Syntax: playuntildone(sound_name = “Grunt”)

istouching()

The function checks if its sprite is touching the mouse-pointer, edge, or another sprite. If the sprite is touching the selected object, the block returns true; if it is not, it returns false.

Syntax: istouching(object_name = “_edge_”)

stamp()

The function is used when the sprite needs to produce a bitmap image of itself which is stamped onto the stage. (Because it is merely a picture of the sprite and not a sprite itself, it cannot be programmed). The function will not draw over sprites.

Syntax: stamp()

setled()

The function sets the specified RGB LED of Quarky (specified with X and Y position of the LED) to the specified RGB color and brightness value.

Syntax: setled(x_position = 1, y_position = 1, color = [0, 0 , 0], brightness = 20)

readpushbutton()

The function returns the state of the specified push button. If the button is pressed it returns True or else False.

Syntax: readpushbutton(button = “L”)

runrobot()

The function moves the Quarky robot in the specified direction. The direction can be “FORWARD”, “BACKWARD”, “LEFT”, and “RIGHT”.

Syntax: runrobot(direction = “FORWARD”, speed = 100)

playsound()

The function plays the specified audio on the Quarky speaker. The function does not have any callbacks, so other functions can be executed while this function is running.

Syntax: playsound(audio = “QuarkyIntro”)

video()

This function helps turn the video on/off on the stage with a defined level of transparency.

Syntax: video(video_state = “on”, transparency = 1)

video()

This function helps turn the video on/off on the stage with a defined level of transparency.

Syntax: video(video_state = “on”, transparency = 1)

setthreshold()

The function sets a loudness filter threshold to remove the background noise from the audio file which is being analyzed.

Syntax: setthreshold(loudness = 30)

initialisepickplace()

The function initializes the pick and place robot with the specified orientation.

Syntax: initialisepickplace(orientation = “HORIZONTAL”)

tf.keras.models.load_model()

The function loads a model saved via model.save().

Syntax: tf.keras.models.load_model(filepath = ‘saved_model.h5’, custom_objects = None, compile = True, options = None)

Quadruped()

The function initializes the quadruped robot object in Python and maps the 8 servos to the specified pins.

Syntax: Quadruped(Front Right Hip = 4, Front Left Hip = 1, Front Right Leg = 8, Front Left Leg = 5, Back Right Hip = 3, Back Left Hip = 2, Back Right Leg = 7, Back Left Leg = 6)

setrelay()

The function set the state of the relay connected to the selected pin to High or Low. A high state means that the pin will have 3.3V and for Low, the pin will be 0 V.

Syntax: setrelay(state = 1, pin = “D3”)

wifi.connecttowifi()

The function connects the Quarky or ESP32 to the specified Wi-Fi and password. The block is only available in the Upload Mode when the code is uploaded to Quarky.

Syntax: wifi.connecttowifi(WIFI = “Wi-Fi Name”, PASSWORD = “password”)

wifi.iswificonnected()

The function reports if the Wi-Fi is connected to the Quarky or ESP32 or not. This block is only available in Upload Mode.

Syntax: wifi.iswificonnected()

MarsRover()

The function initializes the Mars Rover object in Python and maps the 5 servos to the specified pins.

Syntax: MarsRover(Head = 4, Front Left = 1, Front Right = 7, Back Left = 2, Back Right = 6)

Humanoid()

The function initializes the humanoid robot object in Python and maps the 6 servos to the specified pins.

Syntax: Humanoid(Right Hip = 7, Left Hip = 2, Right Foot = 6, Left Foot = 3, Right Hand = 8, Left Hand = 1)

Mecanum()

The Quarky Mecanum Robot Drive Motors are initialized by the function, which assigns each motor to a specific port. This allows the robot to be programmed to move the motors in the desired direction.

Syntax: Mecanum(Front Left = 1, Back Left = 2, Back Right = 7, Front Right = 8)

initexpansion()

The function initializes the Expansion Board of Quarky for use. Without initialization, the board will not respond to any other functions.

Syntax: initexpansion()

math.acos()

The function returns the arc cosine of a number.

Syntax: math.acos(x)

roboticArm.calibrate()

This function performs the calibration process for a robotic arm by taking in the error angles of each of its three servos (link 1, link 2, and base). It then stores the offset angle for each servo to the memory of the Quarky, ensuring that the robotic arm is functioning correctly at all times.

Syntax: roboticArm.calibrate(Link1 Offset = 0, Link2 Offset = 0, Base Offset = 0)

AdvanceLineFollowing()

This function ensures proper sensor alignment for precise line-following control.

Syntax: AdvanceLineFollowing(ir_num = 2)

setdigitaloutput()

The function sets the digital state of the specified pin to HIGH or LOW / 0V or 3.3V.

Syntax: setdigitaloutput(pin, state)

enablebox()

The function enables the automatic display of the box on face detection on the stage.

Syntax: enablebox()

All articles loaded

No more articles to load

Object Detection

Introduction

Accessing Object Detection in Block Coding

Accessing Object Detection in Python Coding

PictoBlox Blocks

Block Coding Examples

Python Functions

Python Coding Examples

Company

Community

Get in Touch

Follow Us

Dabble App - One App. Infinite Control.