A project for all ages with an interest in Artificial Intelligence by team Autobots
AutoBots is a maze puzzle where students get to learn how image recognition works with robots and how proper data training is essential for robots to recognize images.
This workshop has participants train their own Convolutional Neural Network to recognize custom hand gestures and allow control of a Makeblock Mbot Mega. Students get to create their own distinct dataset based on images captured of their hand. The robot would then navigate through some obstacles and each gesture would reflect how well the robot could recognize those images.
Click the image below to watch a 30 second commercial for the project
- Difficulty Level: Intermediate
- Target Audience: 6-12, College, Non-Stem
- Duration of Workshop: 1 hour
- Needed Materials: Mbot Mega, Google Chrome web--browser, Laptop with camera
- Learning Outcomes: The primary goal is to show the audience how distinct training datasets can effect how well a neural network can recognize images. Specifications such as lighting and angle of the image can impact the accuracy of the model.
- Your Main Technology MBlock (Uses Scratch Block Coding)
- Technology Ambassador Program (TAP) is a project-based class that provides a collaborative environment for students to work with their fellow classmates on a semester-long project using technologies of their choice. TAP strives to increase participation in IT through numerous outreach activities and workshops that are designed to showcase the creative and fun side of technology.
(From left to right: Jonathan, Krishan, and Isaiah)
- Jonathan Tran
- Krishan Bhalsod
- Isaiah Gorman
- Dr. Cengiz Gunay
- Dr. Cindy Robertson
In this project, students will design custom hand gestures or images to represent specific movements or commands for an MBot robot. Using image recognition and machine learning, students will train a model to recognize these gestures and teach the robot to react accordingly—whether it’s moving forward, turning, stopping to navigating obstacles. This hands-on project will allow students to develop their skills in image recognition, AI model training, and robotics control, while exploring the complexities and challenges of creating real-world applications using AI and robotics technologies.
Our Demo Video Click on the image
- R. Yamashita, M. Nishio, R. K. G. Do, and K. Togashi, “Convolutional neural networks: an overview and application in radiology.,” Insights Imaging, vol. 9, no. 4, pp. 611–629, Aug. 2018, doi: 10.1007/s13244-018-0639-9.
- Tapia Abstract - Bhalsod, K., Gorman, I., Tran, J., Gunay, C., & Robertson, C. (2025, April 11). Learning Convolutional Neural Networks Using mBots.
- STARS and CREATE Symposium - Bhalsod, K., Gorman, I., Tran, J., Gunay, C., & Robertson, C. (2025, April 11). Controlling Robots Using Image Recognition.
List the outreach events your team has participated in.
- Atlanta Science Festival March 15, 2025, Georgia Gwinnett College: Promote IT field and teach Convolutional Neural Networks to those interested.
- Atlanta Science Festival March 22, 2025, Piedmont Park (Atlanta): Promote IT field and teach Convolutional Neural Networks to those interested.
- Class Workshops March 31 & April 1 & 4, 2025, Georgia Gwinnett College: Promote IT field and teach Convolutional Neural Networks to those interested.
- Jedi (Sphero)
- Ballislife (Sphero)
- DancingSphero (Sphero)
- Code-Inator (MBot)
- Wanda (Cosomo)
- TechGoldFish (Sphero)
- Dronereality (Drones)
If you're interested in more workshops that utilize Scratch, check out Space Mechanic!
- MBlock is a block-based programming languaged used to teach kids and beginners about coding. It's based off the Scratch 3.0 interface.
- Because of it being based off Scratch, beginners don't need to worry about coding due its drag-and-drop interface. The mBlock software contains a plethora of extentions and it allows support for AI.
- Autobots uses MBlock, which is a Scratch program that comes along with the MBot. While MBlock is Scratch-based, it also contain Google's teachable machine, which helps train the MBot in image recognition.
Building the mBot Mega Instructions
- Click on the link here: https://planet.mblock.cc/project/5574657
- Turn on MBot
- Scroll down and click source code
- On the device tab, select bluetooth, enable bluetooth if prompted, and connect.
- If done right, the blue light within the MBot will stop blinking.
Click here to view instructions
Post connection to the MBot with MBlock open with the correct link. WEBCAM AND GOOGLE CHROME IS REQUIRED.
- On the sprite tab, scroll down from the open bar where all code blocks are stored.
- If done correctly, there is a block for teachable machine. Click on it.
- On top of the block codes, there is a Create/Manage model. Click on it.
- Scroll down and select Image Recognition (or alike).
- Ensure that webcam is enabled/turned on. Create desired amount of classes.
- Using the webcam, take a single or multiple pictures of your desired object. In this case, hands would be the object.
- Do this for all classes. Ensure that each object is distinct from each other.
- When finished, change model settings to your desired preference. For this project, keep settings default. Click on train model.
- When finished, webcam should turn on. Test your objects to ensure right classes are being identified.
- If correct, click on the top left to use model.
- While on the sprite tab, replace all instances of model name and classes to your model/class naming convention.
- Run code to test if models and classes are working. If it is not working, check if all instances are correctly placed (i.e. Stop function is replaced with Stop class equivalent). Otherwise, you just made your first Convolutional Neural Network model!
Click here to view workshop walkthrough PDF file
Workshop Video on how to control your mBot Click on the image
