Research on Vision-based Gesture Recognition

2021 COE Engineering Design Project (XZ03)

---

Faculty Lab Coordinator

Xiao-Ping Zhang

Topic Category

Embedded Systems

Preamble

In human-computer interaction, gesture recognition is essential. The gesture database is established through vision and the model is established for gesture classification and recognition.

Objective

Hand detection and segmentation from a video. Hand tracking. Hand feature, such as pixels and skeleton. Deep learning theoretical knowledge. Design a model based on neural network for gesture recognition.

Partial Specifications

Collect/generate videos with RGB data, with multiple labels of gestures.
Select the proper feature for recognition.
Design a neural network model can handle time series well.

Suggested Approach

Learn deep learning models for gesture recognition.
Design an end-to-end framework, from an input video, hand detection, hand tracking, to classification result.
Collect data with different labels and learn the features like pixel or skeleton as input of the models.

Group Responsibilities

Coordinate to define software interfaces and to divide tasks. Realize and demonstrate the entire software system. Deliver the design documents for each module and interfaces.

Student A Responsibilities

Learn deep learning knowledge.
Collect/generate RGB video data, discuss with all the students.
Design hand detection algorithms with deep learning methods.
Collaborate with Student B for hand detection and tracking.

Student B Responsibilities

Learn deep learning knowledge.
Collect/generate RGB video data, discuss with all the students.
Design hand tracking algorithms with deep learning methods.
Collaborate with Student A for hand detection and tracking.

Student C Responsibilities

Learn deep learning knowledge.
Design gesture recognition algorithms with deep learning methods.
Collaborate with Student D for gesture recognition.
Integrate hand detection and tracking module and gesture recognition module into a unified framework.
Coordinate with the other student to divide tasks. Need to clearly indicate what parts of tasks have been done in the final report.

Student D Responsibilities

Learn deep learning knowledge.
Design gesture recognition algorithms with deep learning methods.
Collaborate with Student C for gesture recognition.
Integrate hand detection and tracking module and gesture recognition module into a unified framework.
Coordinate with the other student to divide tasks. Need to clearly indicate what parts of tasks have been done in the final report.

Course Co-requisites

To ALL EDP Students

Due to COVID-19 pandemic, in the event University is not open for in-class/in-lab activities during the Winter term, your EDP topic specifications, requirements, implementations, and assessment methods will be adjusted by your FLCs at their discretion.