Gesture Control of Drone Using a Leap Motion Controller

2021 COE Engineering Design Project (TY01)


Faculty Lab Coordinator

Truman Yang

Topic Category

Software Systems

Preamble

The leap motion controller is a small device that tracks hand and finger movements using infrared LEDs, allowing users to input gesture commands into an application in place of a mouse or keyboard. In this project, a leap motion controller will be utilized to control a drone through simple human gestures. The hand gestures captured by the LEAP will be interpreted,and transmitted to the drone in order to control its motions via these gestures. Students need to provide two kinds of solutions: AI-based or the solutions do not need the support from AI.

Objective

(1) Identify possible gestures that can be detected by the gesture capturing device, then assign them to drone’s functions. Your gesture control should be able to fulfill all drone’s functions. (2) Literature review will be conducted. Design and implement two to three AI-based gesture recognition models, based on existing neural network models/Machine learning techniques to uncover the best approach for drone control. Further, make improvement on the best approach you determined and evaluate them.

Partial Specifications

(1) Evaluate two to three AI-based models/Machine learning techniques over hand-gesture images and the approach in terms of their accuracy, precision, and complexity. The solutions that do not need AI support will also be evaluated.
(2) Algorithms should be effectively and efficiently.

Suggested Approach

(1) Software development in Python.
(2) Your design and implantation could be based on existing models and should make some improvements.
(3) Idea generation technique with Scamper

Group Responsibilities

(1) Programming on LEAP and the drone
(2) Design, implement, and document the software as specified above.

Student A Responsibilities

Design and implement the solution that does not need neural network support.

Student B Responsibilities

Design and implement the first AI-based gesture recognition model with python.

Student C Responsibilities

Design and implement the second AI-based gesture recognition model with python.

Student D Responsibilities

Evaluate the performance of different solutions.

Course Co-requisites

COE318: Software Systems Introduction to Image Processing

To ALL EDP Students

Due to COVID-19 pandemic, in the event University is not open for in-class/in-lab activities during the Winter term, your EDP topic specifications, requirements, implementations, and assessment methods will be adjusted by your FLCs at their discretion.

 


TY01: Gesture Control of Drone Using a Leap Motion Controller | Truman Yang | Sunday September 5th 2021 at 05:13 PM