Development Projects
TripBot - Conversational Interface to Provide Actionable Independent Travel Insight to Blind and Low Vision (BLV) People (Jan. 2024 - Present)
- Integrated LLMs into a conversational agent providing accurate environmental condition information to BLV individuals, enhancing independent travel insights.
- Fine-Tuning and leveraged LLaVA to provide continuous instruction and Orientation and Mobility (O&M) aid to BLV individuals, improving their navigation and mobility skills.
- Conducting pilot testing with BLV people to assess system robustness and fine-tune based on user feedback, using a preference strength algorithm to incorporate BLV individuals’ perceptions of LLM responses.
Investigating the Potential of Immersive Technologies in Spatial Learning for Blind and Low Vision Individuals (Sept. 2023 - Present)
- Prototyped a VR ping-pong game encouraging play between sighted and BLV individuals, promoting physical activity and cognitive engagement through gameplay.
- Led the development of dynamic and spatial sound for the virtual ball, based on force, trajectory, and proximity, enhancing the immersive experience.
- Generated a novel haptic feedback mechanism using a haptic glove, varying intensity and frequency to provide an additional mode of information transmission, improving user interaction.
RLHF Voice-Based Conversational Agent on Intelligent Reflective Surfaces (Mar. 2023 - Jan. 2024)
- Conceptualized and engineered a smart mirror with a voice agent, pioneering mood analysis via emotion recognition and speech feature analysis using Python, JavaScript, and Node.js, enhancing user emotional well-being.
- Implemented an RNN model for speech analysis and integrated a novel RLHF algorithm based on POMDP, dynamically responding to user emotions, improving real-time interaction accuracy.
- Orchestrated the technical creation and empirical research, devising the conversational interface and examining the dynamics between emotion recognition, cognitive engagement, and attention metrics, providing insights for improving user experience.
- Integrated the Mistral and LLaMA-uncensored model into the emotion agent, catering to a vector space of emotions and providing low-risk advice for mood alleviation, increasing user satisfaction.
- Created a system with a strong capacity to motivate users succinctly, currently addressing hallucination issues related to hobbies and aiming to eliminate them based on user interaction studies.
IoT-Based Door Locking System in a Smart Living Environment (Mar. 2022 - Dec. 2022)
- Developed an advanced door locking system that recognizes the user’s face for automatic opening, enhancing accessibility and convenience.
- Initiated the creation of a guidance robot for the Blind and Visually Impaired (BVI), which understands unknown environments and provides navigational guidance, improving independence for BVI individuals.
M. HOSTELS (Dec. 2021 - Jun. 2022)
- Developed a Dynamic Pricing Algorithm through parametric articulation to set prices for rooms based on time, date, and nature of booking, optimizing revenue management.
- Created digital wireframes and high-fidelity prototypes to initiate development, enhancing the design process and user experience.
AI-Based Natural Disaster Analysis System - IBM Externship (Jul. 2021 - Oct. 2023)
- Won Best Project title for creating a flask application/CNN model for image analysis and comparison with a database to predict the presence of natural disasters, improving disaster response and management.
OPTiMA (Apr. 2021 - Jun. 2021)
- Defined the concept for interface design and created digital prototypes, ensuring the inclusion of accessibility features for the BVI, enhancing usability.
- Produced a working android application for timeslot distribution, streamlining scheduling processes.
Research Projects
Autistic Education Study (Mar. 2023 - Jun. 2023)
- Created a fine motor skills development, ethics, and behavior learning application for autistic children, enhancing their learning experience.
- Conducted user research and systematic literature review (SLR) on current methods employed for autistic learning, informing the development of effective educational tools.
- Interviewed autistic students to understand their learning processes, guiding the design of technological interventions to better meet their educational needs.
Role of Upbringing, Education, Culture, and Society in Shaping Opinions (Mar. 2022 - May. 2022)
- Conducted data cleaning, formatting, and visualization on the Open-Source General Social Survey, improving data quality and presentation.
- Identified trends and correlations between ethnicity, geographical location, and overall happiness, as well as the influence of factors such as gender, degree, family size, and marital status on reported happiness, providing insights into social dynamics and well-being.
Hardware Projects
Smart Shoe (Mar. 2023 - Present)
- Measured the pressure exerted on each foot to identify potential knee issues, enhancing injury prevention and user comfort.
- Factored in the effect of injury by measuring differential weight exerted by each foot, providing insights for improving user health.
- Created the entire software design for the shoe, including a deep learning algorithm capable of dynamically shifting foot pressure based on user stance, and designed the basic hardware circuit.
OCTABIN (Mar. 2022 - Jun. 2023)
- Led the team in application development, generating software with an inbuilt reward system to encourage waste management and pinpoint bin locations, promoting environmental sustainability.
- Worked on the hardware for the bin, implementing dynamic spacing and sorting mechanisms, improving waste management efficiency.
ViBrain (Jun. 2021 - Aug. 2021)
- Conducted research to develop a device for the Deaf and Hard-of-Hearing (DHH) using bone conduction technology, reducing lifestyle gaps and enhancing communication capabilities.