Machine Learning
•
Artificial Intelligence
•
Tangible Design
Mar 2022
School Of Sand
School Of Sand
Accepted to the Student Design Consortium @ IndiaHCI 2023
Accepted to the Student Design Consortium @ IndiaHCI 2023
✨Project Highlight
✨Project Highlight
The team presented this project to the Hon'ble Governor and Cabinet Minister of Uttarakhand & gained their support at UPES Sustainability Fair 2022.
The team presented this project to the Hon'ble Governor and Cabinet Minister of Uttarakhand & gained their support at UPES Sustainability Fair 2022.
Overview
Bringing a tactile interface to primary school classrooms to improve the initial learning and teaching experience for students, especially those with developmental delays, and teachers.
15+
Papers Reviewed
Timeline
Timeline
March 2022 - Ongoing
March 2022 - Ongoing
My Role
My Role
Mainly focusing on the user experience of the product as well as the AI/ML software.
Mainly focusing on the user experience of the product as well as the AI/ML software.
Tools
Tools
mBlocks
Adobe Illustrator
AutoDesk Fusion 360
mBlocks
Adobe Illustrator
AutoDesk Fusion 360
Overview
Bringing a tactile interface to primary school classrooms to improve the initial learning and teaching experience for students, especially those with developmental delays, and teachers.
15+
Papers Reviewed
Overview
Bringing a tactile interface to primary school classrooms to improve the initial learning and teaching experience for students, especially those with developmental delays, and teachers.
15+
Papers Reviewed
I'm interactive!
The Background
The Background
Teaching language to young children is a difficult task, a part of that issue is based on the fact that before they are taught to write and read, they are taught to hold a tool for writing. Be it chalk, pencil or a stick, children have always been taught to write using tools. The alleviation of this need has been the focus of our project. Our idea works on the basis of giving children a more tactile medium to learn languages. Using their own physical bodies means they feel more connected with what they are doing and this results in them having better information retention regarding the same.
The proof of concept was inspired by traditional sand artists, and aims to apply the action performed by them while drawing on sand to allow children to create shapes in sand without having to use any tool, keeping the experience tactile and attached for the child.
Teaching language to young children is a difficult task, a part of that issue is based on the fact that before they are taught to write and read, they are taught to hold a tool for writing. Be it chalk, pencil or a stick, children have always been taught to write using tools. The alleviation of this need has been the focus of our project.
Our idea works on the basis of giving children a more tactile medium to learn languages. Using their own physical bodies means they feel more connected with what they are doing and this results in them having better information retention regarding the same.
Teaching language to young children is a difficult task, a part of that issue is based on the fact that before they are taught to write and read, they are taught to hold a tool for writing. Be it chalk, pencil or a stick, children have always been taught to write using tools. The alleviation of this need has been the focus of our project.
Our idea works on the basis of giving children a more tactile medium to learn languages. Using their own physical bodies means they feel more connected with what they are doing and this results in them having better information retention regarding the same.
Teaching language to young children is a difficult task, a part of that issue is based on the fact that before they are taught to write and read, they are taught to hold a tool for writing. Be it chalk, pencil or a stick, children have always been taught to write using tools. The alleviation of this need has been the focus of our project.
Our idea works on the basis of giving children a more tactile medium to learn languages. Using their own physical bodies means they feel more connected with what they are doing and this results in them having better information retention regarding the same.
The proof of concept was inspired by traditional sand artists, and aims to apply the action performed by them while drawing on sand to allow children to create shapes in sand without having to use any tool, keeping the experience tactile and attached for the child.
The proof of concept was inspired by traditional sand artists, and aims to apply the action performed by them while drawing on sand to allow children to create shapes in sand without having to use any tool, keeping the experience tactile and attached for the child.
Traditional Indian sand art, one of the inspirations of this project. (Photo & Artwork Credits – Sand Artist Badal Barai)
Traditional Indian sand art, one of the inspirations of this project. (Photo & Artwork Credits – Sand Artist Badal Barai)
The Vision
The Vision
The child engages with the sandbox using their hand on Indian art sand (rangoli) in a bottom-illuminated sandbox. This writing is then read by a cognitive vision model (currently running on Google Teachable Machines). Upon recognizing a valid input, the system then outputs a slide for the written alphabet, with images and an auditory output to stimulate multiple senses, explaining the use of that alphabet in multiple languages.
The child engages with the sandbox using their hand on Indian art sand (rangoli) in a bottom-illuminated sandbox. This writing is then read by a cognitive vision model (currently running on Google Teachable Machines). Upon recognizing a valid input, the system then outputs a slide for the written alphabet, with images and an auditory output to stimulate multiple senses, explaining the use of that alphabet in multiple languages.
Our design aims to be a low-cost, sustainable and scalable medium that runs on low-cost hardware and does not require expensive and potentially dangerous equipment like a top mounted projector. Furthermore, the input and especially the output in our prototype stimulate multiple senses to improve the redundancy of the information being perceived by the learner, which improves retention and learning effectiveness.
Our design aims to be a low-cost, sustainable and scalable medium that runs on low-cost hardware and does not require expensive and potentially dangerous equipment like a top mounted projector. Furthermore, the input and especially the output in our prototype stimulate multiple senses to improve the redundancy of the information being perceived by the learner, which improves retention and learning effectiveness.
A diagrammatical representation of the design alongside the physical prototype.
A diagrammatical representation of the design alongside the physical prototype.
The Execution
The Execution
This project was executed in three steps: The Sandbox, the Code and the Output
This project was executed in three steps: The Sandbox, the Code and the Output
The Sandbox
The Sandbox
We designed and built a sandbox that mimics the boxes used by sand artists to provide a glow underneath the sand. This was done to give the computer vision model a fixed background so that the only variable it has to deal with is what is written in the sand.
We designed and built a sandbox that mimics the boxes used by sand artists to provide a glow underneath the sand. This was done to give the computer vision model a fixed background so that the only variable it has to deal with is what is written in the sand.
The process of laser cutting the parts & assembling the box.
The process of laser cutting the parts & assembling the box.
Read more about the Sandbox
The Code
The Code
Google Teachable Machines is used on Mblocks, a code-block platform, to prototype the ML model. The software is currently running on a Windows laptop, which allows the computer vision model to process color images for recognition.
Google Teachable Machines is used on Mblocks, a code-block platform, to prototype the ML model. The software is currently running on a Windows laptop, which allows the computer vision model to process color images for recognition.
The ML Model and Logic behind the system.
The ML Model and Logic behind the system.
Read more about the ML Model
The Output
The Output
The output depends on the info provided by the ML model. It consists of a slide informing the user about the letter they have just drawn, for example, if the user enters the letter “A”, the screen displays an image of an apple, the letter A, and the sentence “A for Apple”, presented on an illustrated backdrop. The software also plays audio repeating the sentence written on screen in both English and Hindi. All of this – the sentence shown, the audio played and the illustrations used, can be configured to fit the context and region of the child.
The output depends on the info provided by the ML model. It consists of a slide informing the user about the letter they have just drawn, for example, if the user enters the letter “A”, the screen displays an image of an apple, the letter A, and the sentence “A for Apple”, presented on an illustrated backdrop. The software also plays audio repeating the sentence written on screen in both English and Hindi. All of this – the sentence shown, the audio played and the illustrations used, can be configured to fit the context and region of the child.
The explainer video submitted to IndiaHCI 2023. Watch to better understand the design.
The explainer video submitted to IndiaHCI 2023. Watch to better understand the design.
What's Next?
What's Next?
While we built & programmed the project, we had several design observations and learnings to apply towards the next iteration of the prototype.
The prototype was also tested on children at a nursery, with an age range consisting of 2-8 years of age.
While we built & programmed the project, we had several design observations and learnings to apply towards the next iteration of the prototype.
The prototype was also tested on children at a nursery, with an age range consisting of 2-8 years of age.
Photos from the usability testing session, looking at the response of the children and the accessibility to the teacher.
Photos from the usability testing session, looking at the response of the children and the accessibility to the teacher.
Here are the changes we came up with based on our observations:
Here are the changes we came up with based on our observations:
• The user flow of the device has been altered to better facilitate teacher-student interaction as well as improve the teaching capability of the design.
• The user flow of the device has been altered to better facilitate teacher-student interaction as well as improve the teaching capability of the design.
• The overall form of the sandbox was also explored to improve the accessibility of the sand for the child, and also improving the methods for replacing sand, servicing and general maintenance.
• The overall form of the sandbox was also explored to improve the accessibility of the sand for the child, and also improving the methods for replacing sand, servicing and general maintenance.
• The data set for each character will be recreated in black and white to simplify processing. More characters will be included per character to improve accuracy and read timings.
• The data set for each character will be recreated in black and white to simplify processing. More characters will be included per character to improve accuracy and read timings.
• The illustrations and text used in the slides will be recreated to be more child friendly and more options will be included for configuration based on locale and language.
• The illustrations and text used in the slides will be recreated to be more child friendly and more options will be included for configuration based on locale and language.
This project is in active development🧑💻
✨Stay Tuned for Updates!
This project is in active development🧑💻
✨Stay Tuned for Updates!
Connect to Content
Add layers or components to infinitely loop on your page.
Built with
and lots of
Back To Top
Copyright © Param Dadegaonkar