HackAPrompt
Applied ResearcherFeb 2023 - May 2024
Why
~ The Smithsonian American Art Museum struggled to help the visually impaired navigate the galleries independedently.
Goal
~ AR Hologram that helps the visually impaired with sensory subsititutes to understand artworks.
~ AR Hologram that helps the visually impaired with sensory subsititutes to understand artworks.
Tech
~ Microsoft HoloLens 2
~ Meta Quest 3
~ Unity
~ Blender
~ Illustrator
~ Reaper Daw
~ Microsoft HoloLens 2
~ Meta Quest 3
~ Unity
~ Blender
~ Illustrator
~ Reaper Daw
The Process
Research   ->   Ideation   ->   Final Designs   ->   Reflection
Research
Visually Impaired
Smithsonian Visitors
Art Enthusiasts
User Interviews
Museum Documentation
Mixed Reality Toolkit
Key Findings
The headset would help guide the user throughout the exhibits.
Sensory subsitution such as haptics would help those understand the pieces beyond audio descriptions.
The user can immersive themselves at their own pace and doesn’t require a tour guide.
User InsightsÂ
About Maria
Age: 34 years old
Occupation: Teacher (visually impaired)
Education: Master’s Degree
Technology Use:
Internet: High
Using Software: Medium
Using Mobile Apps: High
Age: 34 years old
Occupation: Teacher (visually impaired)
Education: Master’s Degree
Technology Use:
Internet: High
Using Software: Medium
Using Mobile Apps: High
Motivations:
~ Wants to independently enjoy art and cultural spaces without relying on others.
~ Interested in experiencing artworks through multisensory approaches.
~ Seeks tools that empower accessibility and inclusivity.
Challenges:
~ Traditional museum guides rely heavily on visuals, creating barriers.
~ Needs assistive technologies that are intuitive and easy to adopt.
~ Overwhelmed by overly technical or poorly designed accessibility tools.
Opportunities:
~ AR holograms can provide tactile, auditory, or descriptive overlays.
~ Technology can bridge independence gaps and make the museum more welcoming.
~ Customizable features (e.g., voice speed, tactile feedback) enhance personal control.
About James
~ Age: 68 years old
~ Occupation: Retired, Veteran
~ Education: High School
Technology Use:
~ Internet: Low
~ Using Software: Low
~ Using Mobile Apps: Medium
~ Age: 68 years old
~ Occupation: Retired, Veteran
~ Education: High School
Technology Use:
~ Internet: Low
~ Using Software: Low
~ Using Mobile Apps: Medium
Motivations:
~ Wants to stay connected to cultural experiences despite visual impairment.
~ Desires activities that provide cognitive engagement and reduce isolation.
~ Enjoys sharing experiences with family members, especially grandchildren.
Challenges:
~ Steep learning curve for new devices (e.g., HoloLens, AR headsets).
~ Limited confidence in navigating galleries independently.
~ May be hesitant to adopt new tech without strong support.
Opportunities:
~ Simplified onboarding and voice-based instructions can reduce intimidation.
~ Group features (e.g., shared AR experiences) can encourage family participation.
~ Technology can reduce reliance on staff, giving a greater sense of dignity.
Product Design Requirements
Using Digital Scans to section the different 3D components.
Specific descriptions of where the user is pointing at to explain the details.
Enabling the user to intuitvely understand where everything is.
Prototype
Impact
 Spatial Sense aims not only to enhance visually impaired visitiors experience at the Smithsonian but also to raise awareness to artificat preservation as the experience is not limited to the physical museums.
 Spatial Sense aims not only to enhance visually impaired visitiors experience at the Smithsonian but also to raise awareness to artificat preservation as the experience is not limited to the physical museums.
Research Team: Ian McDermott, William Le, Ben Margolin, Eva Ginns, and myself