|
Self-supervised Learning of Motion Capture, NIPS2017, Spotlight
(website)
Mar. 2017 – Dec. 2017
- Invented an end-to-end motion capture neural network model to recover 3D human mesh from a single monocular video using Python and Tensorflow.
- Pre-trained the model with synthetic data and fine-tuned it with self-supervised losses on real world video.
- The 3D mesh prediction result on H3.6M real video dataset reduced the per-joint error rate by 83.4%.
|
|
Face Motion Prediction by Brain-wave Detection, ICORR 2017
Feb. 2017 – Oct. 2017
- Constructed a hand-free, light-weight, wearable device to detected user’s brain-wave using OpenBCI EEG sensors.
- Predicted the user’s face motion as controller by applying machine learning classification algorithms on MATLAB.
Brain-wave Controlled Robot Car, IEEE SMC BCI Hackathon 2017
Oct. 2017 – Oct.2017
- Created a first-person-view manipulator robot car controlled by wearing three OpenBCI EEG sensors, VR headset, and two Myo gesture control armbands that reads muscle’s electrical activity.
- Utilized Face Motion Prediction to control the robot car and applied Myo armband to control the manipulator.
|
|
Video of Face Transfer, CMU Intern
May. 2017 – Aug. 2017
- Transferred faces in an input video towards female, male, young equivalents using Generative Adversarial Network.
- Built a neural network model that made the video of transferred faces smoother by using Python and Tensorflow.
<- original(left), my result(right)
|
|
Future Video Prediction, University of Pittsburgh
Feb.2017 – Apr. 2017
- Modeled an Encoder and Decoder neural network structure and trained the model with frame pairs to predict next video frame using Python and Caffe.
<- original video(left right) and results from three differnce Encoder and Decoder neural network structure we bulit.
|
|
Being On Your Way, University of Pittsburgh
Sep. 2016 – Dec. 2016
- Devised an embedded wearable equipment to lead road for blind people by detecting obstacles nearby and informing the user about the location of the obstacles with Arduino board and ultrasonic sensors.
|