SkyTrack
SpeakEye
My role
Tool
6-7 weeks, fall 23
Figma, Illustrator, Photoshop
restaurant Research, Visual Design, Restaurant Rebranding, Interface
timeframe
introudction
research
Imagine a world where visual impairments are no longer barriers to experiencing life to the fullest. Our innovative AI solution is designed to empower those with visual challenges, such as blindness, low vision, or color blindness, enabling seamless integration into society. With our application, users can gain a clearer understanding of their surroundings and access vital information effortlessly. It's not just an app; it's a companion that opens up a world of possibilities through advanced AI technology, enhancing the quality of life for visually impaired individuals.
Background Research
Despite advancements in technology, visually impaired individuals often find themselves at a disadvantage due to the visual-centric nature of information and environmental interaction. Research indicates that obstacles in accessing real-time data, reading text, and identifying colors can lead to a dependency on others, affecting the self-reliance and confidence of those with visual disabilities.
Goals
We aim to develop an AI solution that enables visually impaired individuals to independently access real-time information, read text, and identify colors through intuitive auditory feedback, enhancing their day-to-day life.
Yes
60.5%
Sidelined
39.5%
pay for a life-improving AI?
Yes
92.1%
I am not interested.
7.9%
AI that works with devices like smart glasses?
No
86.8%
Yes
5.3%
Indeterminate
7.9%
AI tools to help with visual impairment?
Yes
73.7%
No
10.5%
Indeterminate
15.8%
AI describing areas help?
persona
user journey map
solution
platform
task flow 1
task flow 2
task flow 3
research summary & insights
The user learns about the AI app through a social media platform, recommendation from a friend, or a community support group for the visually impaired.
How did you first hear about our AI tool for visual impairment?
The user downloads the app from the app store and goes through the voice-guided setup process, which includes a tutorial on how to use the main features.
How straightforward was the process of finding and downloading our AI tool?
1
1
The user begins to integrate the app into daily life, using voice commands to read texts, identify objects, and navigate spaces. They enjoy the intuitive voice feedback and the app's learning curve is short.
What features of the AI tool do you use most often, and why?
1
1
Satisfied with the app's impact on their daily life, the user recommends it to other visually impaired friends and leaves a positive review in the app store.
What specific experiences with the tool prompted you to share it with others?
2
2
survey
❝
Sarah
❝
Sarah is 28 and works as a Test Engineer in San Francisco. She has been dealing with a vision condition called retinitis pigmentosa since she was a teenager. This hasn’t stopped her; it’s made her a smart user and advocate for technology that everyone can use easily.
Every day, Sarah uses tools like screen readers and voice commands to help her use her phone and computer without trouble. She also uses smartwatches, which are very important because they keep her connected and informed all day.
Sarah loves technology because it helps her stay independent. She believes technology should help everyone have the same opportunities.
She says, "Technology has really changed things for me. It makes everything fair, and I always look for new tools that make life easier."
Age: 28 years old
Location: San Francisco
Occupation: Test Engineer
We are crafting an AI-powered app designed to assist users with visual impairments by audibly narrating text from various sources, describing colors with clarity, and providing rich, descriptive weather updates to convey not just data but the ambiance of their surroundings, all aimed at facilitating informed decisions and easier daily navigation.
Text-to-Speech Conversion
AI can now reliably convert text from books, signs, and screens, allowing for greater independence in accessing written information.
Object and Facial Recognition
Visual AI helps users identify objects and people around them, enhancing social interactions and spatial awareness.
Color Identification
Advanced algorithms can describe colors in detail for those with color vision deficiencies, aiding in daily tasks like clothing selection.
ideate
design
How might we make an app that easily scans and speaks text aloud to help people with weak eyesight?
How might we design an app that tells colorblind users the colors they're looking at to help with daily choices?
How might we create an app that describes the weather in detail using sound for people who can't see?
I am going to select a mobile app as the platform for my design solution.
A mobile app tailored just for them. This app will scan barcodes, speaking out detailed product info, nutritional facts, and even user reviews. It's about making shopping straightforward and informed, even without perfect sight. Every detail, every feature, crafted with the visually impaired in mind.
❝
❝
Task: SpeakEye (AI App) for Reading Text Aloud for Low-Vision Users.
SpeakEye is a user-friendly mobile app tailored for individuals with low vision. Leveraging advanced AI technology, the app scans, recognizes, and audibly conveys text from any source, ensuring users can access information effortlessly without eye strain.
As a low-vision individual, user wants the app to scan and read aloud text from documents, signs, or screens so that I can access information without straining my eyes.
Task: SpeakEye (AI App) for Color Identification for Colorblind Users.
SpeakEye is an innovative mobile application built for colorblind individuals. By integrating AI-driven color detection, the app helps users understand and comprehend colors in their surroundings, aiding them in making everyday decisions.
As a colorblind person, user wants the app to identify and describe colors for me so that I can make informed decisions, whether it's choosing clothes or understanding graphics.
Task: SpeakEye (AI App) for Detailed Auditory Weather Descriptions for Blind Users.
SpeakEye provides a blind user with a holistic understanding of the day's weather, ensuring they can prepare and plan their day with confidence using the app.
User wants a rich auditory description of the weather, including not just temperature but also the feel of the day (e.g., "cool breeze with a gentle sun" or "humid with overcast skies")
user testing
(Turn on sound)
EndsEat
Wild Donkey
BACKGROUND
a gateway to the world of AI technologies for visually impaired individuals.
"AI for Visual Impairments" is designed to help individuals who are blind, have low vision, or are colorblind better integrate into society, access information, and improve their quality of life.
conclusion
As AI continues to evolve, its application in assistive technologies promises a future where visual impairment poses less of a barrier to personal and professional endeavors. It is our hope that ongoing innovation and development in this field will lead to even more intuitive, inclusive, and accessible technology solutions.
SpeakEye
Open the SpeakEye app on the phone and log in.
The app provides a clear, easy-to-access button or voice command option to initiate scanning.
(Navigate to the 'Scan' Function.)
The user points their device's camera towards the text they want to scan.
The AI processes the image, identifies the text, and converts it into readable format.
Result:
The low-vision users successfully read and listen text aloud via the app.
The app immediately starts reading the recognized text aloud using a clear voice.
Open the SpeakEye app on the phone and log in.
A prominently displayed button or voice command allows users to initiate the color identification process.
(Navigate to the 'Scan' Function.)
Users can choose to save the color information for later reference.
The user aims their device's camera at the item or graphic whose color they want to identify.
The app describes the color or colors aloud, using easily understandable terms.
For instance, instead of just saying "blue", it might say "sky blue" for better clarity.
The app processes the captured image and recognizes the primary and any secondary colors present.
Result:
The colorblind users successfully recognize and comprehend colors in their surroundings, aiding them in making everyday decisions.
Users have the option to get more detailed information, like shade variations or how common the color is.
(If the app provides a color palette, users can listen to a description of each color in the palette.)
Open the SpeakEye app on the phone and log in.
Upon opening, the user greets the app and asks, "Can you provide with the current weather description?"
Once they have all the information they need, users can opt to close the app or inquire about another location's weather.
After the primary description, the app asks, "Would you like more detailed information or forecast for the day?"
(Users can inquire about humidity levels, chance of rain, expected temperature shifts, etc.)
The app responds with "Sure" or uses "Power Button" to request the weather information.
The app retrieves real-time weather data based on the user's current location.
The app doesn't just read numerical data; it processes the data to generate a rich, descriptive auditory message. For instance, instead of "25°C, 58% chance of rain.", it might say, "It's a pleasant 25°C outside with a gentle breeze, and there's a 58% chance of rain."
Result:
Users with blindness successfully receive a detailed auditory description of the weather, covering not just the temperature but also the overall feel of the day.
feedback
Users with visual impairments found the Face ID feature to be a challenge, as it required locating the camera, which is not always feasible for blind individuals. The 'Sign In' button and 'Forgot Password' link were not optimally accessible due to their visual design limitations.
solution
To enhance accessibility, we replaced Face ID with a voiceprint authentication system in the new design. This allows blind users to authenticate their identity through voice recognition, eliminating the need to find the camera. The interface has been simplified to reduce cognitive load, with a focus on auditory cues for navigation. The visual elements have been enlarged and contrasted for those with partial sight, and the removal of the 'Forgot Password' function assumes the implementation of a voice-guided password recovery process.
feedback
Users with visual impairments indicated that while the button interface was familiar, it often required precise navigation and could be challenging to locate. The directive "Start taking photos" was clear, but initiating the process was sometimes cumbersome without sight guidance.
solution
The new design removes physical buttons and implements voice control, providing audio prompts to guide the user. The command "Count down to turn on the camera, then point it where you want to scan" allows users to operate the camera hands-free and with verbal cues, making the process more accessible for visually impaired users.
feedback
Users found that while the 'Done' button was a familiar element, it required them to locate and press it, which was not always accessible for individuals with visual impairments. The AI's verbal confirmation "My pleasure." was well-received for indicating task completion.
solution
User testing indicates that the no-button, AI-enhanced design is a significant improvement in terms of accessibility and ease of use. The hands-free operation is particularly beneficial for visually impaired users, allowing for smoother interaction without the need for visual or tactile location of buttons.
#D4FF49
#70FFF6
#02010B
#FCFCFC
FONT
COLOR
HEADER
HEADER 2
HEADER 3
Magna mollis euismod. Aenean eu leo quam. Pellentesque ornare sem lacinia quam venenatis vestibulum.
Inter, 32px
Inter, 48px
Inter, 20px
Inter, 16px
Aa
Inter
Semibold
Thank you for watching.
© 2024 by Yuchen Chen. All rights reserved.