Advancing Remote Training Through a Flexible Glove for Real-Time Tracking and Analysis of Hand and Finger Movements



Table of content

  1. Abstract
  2. Related works
  3. Methodology
  4. Experiment Setup and Implementation
  5. Results and Analysis
  6. Conclusion
  7. Publications
  8. Links


In recent years, remote training has gained significant attention as a means to bridge geographical distances and enable individuals to improve their skills remotely. However, accurately tracking, analyzing, and comparing complex hand and finger movements in real-time poses a critical challenge in remote training, as it is essential for providing precise feedback and facilitating skill development. This project focuses specifically on the application of flexible sensor gloves in surgical training.

Traditionally, assessing surgical technical skills for novice surgeons has relied on evaluations by expert surgeons. However, limitations arise when expert surgeons observe and provide feedback on the skills of novice surgeons due to the difficulty of maintaining a direct line of sight and the need for physical presence.

To address this challenge, this research proposal aims to develop a cutting-edge solution using a flexible glove embedded with nine 9-axis Inertial Measurement Units (IMUs). The glove, equipped with sophisticated algorithms, will enable real-time capture, analysis, and wireless transmission of hand movement data to an end device. A mobile application will be used to provide real-time feedback to surgeons during training.

The outcomes of this research have the potential to transform the way training is conducted, particularly in the field of surgical skills development. By leveraging the capabilities of the flexible glove and advanced algorithms, this project aims to revolutionize remote learning by enabling individuals to reach new levels of skill and proficiency. Additionally, the technology developed in this project holds potential for applications in various fields, including music education, sports coaching, sign language detection, virtual reality gaming, and physical rehabilitation.


Solution Architecture

Solution Architecture

Experiment Setup and Implementation

Flex Glove Implementation

  1. Configured the Pololu USB to Serial Adapter

    Configuring the Pololu USB to Serial Adapter involves setting up the communication parameters of the adapter, such as baud rate and serial port settings.

  2. Uploaded a program to test ESP board

  3. Got sensor data from 9 IMUs

    Obtaining sensor data from 9 IMUs (Inertial Measurement Units) implies collecting information from nine sets of sensors that measure accelerations, gyroscopic rates, and sometimes magnetic fields.

Data Transmission to Mobile App

  1. Dual Core Architecture
    • Use two cores
      • Core 0 - for collecting data
      • Core 1 - for Sending data
    • Use Semaphore Handle for core synchronization
  2. Socket Server and Client
    • ESP32 WROOM acts as a socket server

    • Mobile application act as client

Results and Analysis

Optimizing Data Transmission

  1. Transmission Technology Selection (Wi - Fi vs Bluetooth) When selecting the transmission technology, Focused on
    • Data Transfer rate
    • Power Consumption

    Power Consumption

    Since we display movements real-time, Transmission speed is taken in to priority.

    Transmission Time

    Therefore we chose Wi-Fi as the transmission media

    This is the experimental setup used for that

    Experimental Set up

  2. Moving Average Filter
  1. Sampling Rate Adjustment

Sampling Rate

– Data sent without sampling rate adjustment - 188

– Data sent with sampling rate adjustment - 95

  1. Data Compression
    • Run Length Encoding was used

Transmission Time

3D Visualization

Outcomes and Impact

  1. Functioning Glove
    • Convenient for users
  2. Mobile Application
    • Realtime visualization and feedback
  3. Research paper
    • Optimization techniques of data transmission
    • Implementation of the flexible glove



  1. Semester 7 report