何晓艺/Shawn He
Shanghai, China
xiaoyi.he@outlook.com
Education
Master of Engineering in Electronics Engineering (EE)
2017 - 2020Bachelor of Engineering in Electronics Engineering (EE)
2013 - 2017Work Experience
Senior Video Architect
May 2021 - Present- DLSS frame generation research. Core contributor of Nvidia Smooth Motion
- Video frame generation research
- Efficient frame generation for autonomous driving with Nvidia OFA hardware
- Optical flow acceletor algorithm research and improvement
Algorithm Engineer
Apr 2020 - May 2021- Design, implementation and optimization of video compression adaptive preprocessing engine (FFmpeg+TensorRT)
- • involving technologies such as quality assessment, denoising, enhancement, etc
- • For UGC videos, it saves approximately 10% bitrate at the same quality level
- Familiar with C/C++ implementation of various denoising and enhancement algorithms, and their integration with FFmpeg.
- Participated in assembly optimization of video encoders
- Built a video subjective quality blind testing platform to accelerate video algorithm development and verification.
- Built a video encoder testing visualization platform to accelerate encoder development and algorithm verification.
Research Assistant
2015 - 2019- Advisor: Prof. Weiyao Lin. Focused on Action Recognition, DL-based Video Compression, and Features Compression
- Developed algorithms for lossless skeleton data compression
- Developed algorithms for compressed video enhancement using deep learning
- Participated in intelligent surveillance systems and developed human activity recognition algorithms
Algorithm Engineer Intern
Jul 2019 - Sep 2019- Research and development of super-resolution algorithms
- Applied deep learning techniques to enhance low-resolution images and videos
Intern
Mar 2017 - Sep 2017- Implemented a cross-framework test tool for deep learning frameworks (TensorFlow & Chainer)
- Developed applications for layer accuracy, convergence, and performance testing
- Enabled daily testing based on Jenkins and cluster infrastructure
Research Projects
End-to-end Deep Learning Based Image/Video Compression
2019 - 2020Surveyed on end-to-end deep learning based image/video compression. Implemented novel ideas and experimental verifications.
Lossless Compression for Skeletons Data in Surveillance Videos
2018 - 2019Implemented a lossless compression method for skeletons data in videos based on spatial and temporal correlation. Achieved about 84% compression ratio on test surveillance sequences. Published two papers and had one proposal accepted.
Deep Learning Based Video Compression
2017 - 2018Proposed a novel CNN utilizing partition information in video encoder to enhance compressed videos (deblocking). Achieved about 10% bitrate saving on benchmark sequences. Paper accepted by ICIP 2019 (oral), IEEE Transactions on Multimedia 2020 and won 2nd prize in ChinaMM 2018 challenge.
Intelligent Surveillance
2016 - 2017Established a dataset for human fall detection. Developed a real-time deep learning based fall detection algorithm with over 80% accuracy. Won "Best Demo of the Year" award at Microsoft Research Asia Symposium in 2017.
Publications
Enhancing HEVC Compressed Videos with a Partition-Masked Convolutional Neural Network
He, Xiaoyi, et al.
International Conference on Image Processing (ICIP), 2018 (Oral presentation)• 98 citations
Partition-aware Adaptive Switching Networks for Post-processing in HEVC
Lin, Weiyao, He, Xiaoyi, et al.
IEEE Transactions on Multimedia, 2020 • 61 citations
Key-point Sequence Lossless Compression for Intelligent Video Analysis
Lin, Weiyao, He, Xiaoyi, et al.
IEEE MultiMedia, 2020 • 23 citations
A Multimodal Lossless Coding Method for Skeletons in Videos
He, Xiaoyi, et al.
IEEE International Conference on Multimedia & Expo Workshop (ICME), 2019 • 2 citations
Adaptive Lossless Compression of Skeleton Sequences
Lin, Weiyao, Shinde, TS, Dai, Wenrui, Liu, Ming, He, Xiaoyi, et al.
Signal Processing: Image Communication, 2020 • 2 citations
Patents
Image Illumination Compensation Method and System Based on Tone Shift Estimation and Point-by-Point Tone Mapping
He, Xiaoyi; et al.
Convolutional Neural Network-based In-Loop Filtering Method and System for Video Coding and Decoding
Lin, Weiyao; He, Xiaoyi
Generative Adversarial Network-based In-Loop Filtering Method and System for Video Coding and Decoding
Lin, Weiyao; He, Xiaoyi
In-Loop Filtering Implementation Method Based on Joint Construction and Adaptive Selection of Multiple Networks
Lin, Weiyao; He, Xiaoyi; et al.
Skills & Achievements
Technical Skills
- Programming Languages: Python, C++, Matlab
- Deep Learning Frameworks: TensorFlow, PyTorch, Caffe
- Simulation: Labview
- Research Areas: Computer Vision, Video Compression, Deep Learning
Achievements
- One paper about frame generation is accepted by NVIDIA internal conference (NTech 2024, 200/900+ submissions)
- Second prize in "Deep-learning based Post Processing for Compressed Images" challenge (ChinaMM 2018)
- Best Demo of The Year Award at Microsoft Research Asia Symposium
- Merit Student of Shanghai Jiao Tong University
- National Encouragement Scholarship