Jessica Yin

An underwater soft robotic gripper with a liquid metal sensor skin.

Soft Machines Lab, Carnegie Mellon University

Co-Presented at IEEE International Conference on Intelligent Robots and Systems 2018 (IROS) in Madrid, Spain with Tess Hellebrekers

“Liquid Metal-Microelectronics Integration for a Sensorized Soft Robot Skin” by Tess Hellebrekers, Kadri Bugra Ozutemiz, Jessica Yin, Carmel Majidi

Top 10 National Finalist for Society of Women Engineers Collegiate Research Poster Competition, will present at National Conference in Minneapolis, MN

My Role

Soft circuit fabrication: laser cutting, EGaIn, silicone, microelectronics population

Sensor data visualization (Arduino & Processing)

Test bed design & assembly (SolidWorks & laser cutting)

Gripper bracket design (SolidWorks & laser cutting)

Sensor validation (Arduino)

PCB fabrication: laser cutting, soldering, microelectronics population

Demonstration video

Photography

Background

I started working in the Soft Machines Lab under Professor Carmel Majidi and Tess Hellebrekers (PhD Candidate, Robotics Institute) in the summer of 2017. With the Summer Undergraduate Research Apprenticeship (SURA), I earned academic credit (9 units) for working full time in the lab during that summer.

I have continued working in the lab throughout the school year (Fall 2017: 6 units, Spring 2018: 10 units, Fall 2018: 9 units).

An interactive 3D drawing experience.

Introduction to Physical Computing Term Project

Fall 2017

Skills & Technology

Unity

Arduino

Uniduino (Unity asset for Arduino)

Stepper motor

Leap Motion

Projector

Inspiration

Two technologies that I’ve always wanted to explore have been the Unity Game Engine and Leap Motion Sensor, so I took this final project as an opportunity to do that.

I was very inspired by Ralf Breninek’s project, which was a simple game that used the rotating cube interface to represent portals to different worlds. His project used Arduino, Unity, and Leap Motion to demonstrate his interest in 3D projection mapping.

I was also impressed with Leap Motion’s utility “Pinch Draw”, which was made for integration with a VR headset. As someone who recently started to rely heavily on SolidWorks for visualizing larger projects, I loved the idea of being able to draw in 3D.

Process

I started off by breaking down the project into different sections:

· Arduino/Adafruit Motor Shield + stepper motor

· Leap Motion + Unity

· Unity/Uduino + Arduino

· Projector + Cube

Since I know that I tend to struggle with programming and I’m not familiar with C# (for Unity), I started with working on the components that required programming. Thankfully, I was able to adapt the Pinch Draw program from Leap Motion for use without a VR headset pretty easily by configuring some options.

Adafruit’s motor shield streamlined the process of powering a 12V stepper motor while having it connected to an Arduino– powering components is still something I am not confident with, so I was glad to have it.

The part that I definitely struggled with the most was putting it all together at the end. I spent many hours debugging my code in both Arduino and Unity while trying to make Leap Motion control the Arduino. I had issues with figuring out how to rotate the drawing with the cube,  as well as synchronizing both the virtual and physical cube rotation. I used Uduino, an asset from the Unity Asset Store, to greatly simplify the process in controlling the Arduino through Unity, but I still had to spend hours debugging due to my lack of experience with C-based languages/Unity and restricted access to the Serial port (for print debugging). I honestly did not think that I would finish in time, but miraculously everything started working (for the most part)  in the last three hours before it was due.

Further Work

While I am happy with what I have accomplished so far with this project, there is still a lot of room for improvement. One of the most pressing things I would like to fix is syncing the rotation of the virtual and physical cube. It honestly doesn’t work too well, and that’s a major part of the user experience that I would like to improve. I would also like to improve the form factor– I originally had designed a stand to hide all the wires going everywhere, but didn’t have the time or materials to build it. Some other features that I wanted to include were the ability to change the color of the line drawing by pressing buttons, as well as a reset/new canvas button, as well as the ability to save drawings.

A modular and interactive physical representation of a neural network.

Collaborator: Michael You, Electrical and Computer Engineering 2020

Built at HackPrinceton Spring 2018 (36 hours)

Top 10 Grand Prize Finalist

1517 Fund Prize Winner

My Role

Collaborative ideation

CAD design

Rapid prototyping

User interface design

Laser cutting

Assembly

Pogo pin integration

Soldering

Hardware supplies logistics & planning

Pitch presentation

Inspiration

We were inspired by Tensorflow Playground (playground.tensorflow.org), which offers a visual, interactive way for beginners and experts alike to learn how the inner workings of how neural networks function.

Description

With neural networks as one of the premier models for machine learning, it's no surprise that almost everyone is interested in learning about the model, and figuring out how to apply it to projects. However, for those without a background in computer science, it can be an intimidating and abstract concept to understand how neural networks actually work. For example, much of the data transformations done by a neural network involve complex topology and math that leading scientists still don't understand. Omega^3 overcomes this barrier by offering an intuitive learning experience, using modular blocks and a simple user interface to show all the aspects of how a neural network learns.

Omega^3 works by displaying all views of the neural network training process. The program starts off by asking the user for a dataset to train on, and afterwards, in real-time, incrementally displays the user all the hidden layer data, along with current predictions and test error. By providing all of these views, the user is able to take a closer look at all the inner machinery of neural networks.

In addition, all the blocks are movable, and the program automatically adjusts the neural network structure according to where the blocks are placed. From the experience, users can not only learn how neural networks train to learn new data, but also see limitations of what neural networks can do.

For example, if the user uses too many layers/nodes, they will probably find that their model overfits the data. If the user chooses not enough model complexity, the neural network may show signs of underfitting.

Overall, the option for the user to play around and modify the model as they wish gives them a diverse range of opportunities to discover fascinating and rarely-seen aspects of neural networks.

Technology

-Pybrain, a Python library for machine learning

-Arduino Uno

-Raspberry Pi

-Matplotlib

-SolidWorks

-Laser cutter

-Pogo pins

Stress Analysis Final Project

Collaborators: Julianne Igkobwe (Mechanical Engineering & Physics ‘20), Josh Andah (Mechanical Engineering ‘20)

A virtual reality escape room video game.

Fundamentals of Programming and Computer Science (15-112)

Spring 2017

Skills & Technology

Python

Panda3D (Python library for 3D graphics)

TrinusVR (VR desktop/cell phone app)

DS4Windows (PS4 Controller software for Windows)

Blender

Image editing

Original storyline