Features
Reference
Gesture Recognition
Our gesture-controlled home automation system uses advanced machine learning algorithms to map hand movements into precise control signals, enabling seamless and intuitive smart device automation.
System Architecture
The system integrates hardware and software components for efficient data capture, processing, and execution of smart device commands:
- The MPU6050 sensor captures motion data from wrist movements.
- Data is processed by the ESP32 microcontroller using the Edge Impulse ML model.
- Recognized gestures are sent as MQTT commands to AWS IoT Core, which triggers actions on connected devices like lights, sockets, and door locks.
Key Features
- Hands-free operation with intuitive gestures.
- Dual authentication for door locks (fingerprint and gesture-based).
- Real-time device control and status updates via a Flutter mobile app.
- Secure communication using MQTT and AWS IoT Core.
Hardware Components
| Component | Description |
|---|---|
| ESP32 (NodeMCU) | Processes sensor data, runs the Edge Impulse ML model, and communicates with AWS IoT Core via MQTT. |
| MPU6050 Sensor | Captures precise motion data for gesture recognition. |
| Fingerprint Scanner | Enables secure fingerprint-based door unlocking. |
| Smart Sockets & Light Modules | Controlled via ESP32 to automate appliances and lighting based on gesture commands. |
Software Components
| Component | Description |
|---|---|
| Edge Impulse ML Model | Classifies gestures in real-time on the ESP32 microcontroller. |
| MQTT Communication | Enables reliable message delivery between ESP32 and AWS IoT Core. |
| AWS IoT Core | Acts as the central MQTT broker for managing device commands. |
| Firebase | Provides real-time database updates and role-based access control (RBAC). |
| Flutter Mobile App | Displays real-time device status, logs, and gesture configuration settings. |
System Workflow
- Gesture Captured: MPU6050 records wrist movements and sends data to the ESP32.
- Gesture Classified: Edge Impulse ML model identifies the gesture and maps it to a specific command.
- MQTT Message Published: ESP32 sends the command to AWS IoT Core via MQTT.
- Device Receives Command: Subscribed devices (door lock, smart socket, or light socket) perform the corresponding action.
- Firebase & UI Update: AWS Lambda updates Firebase, and the Flutter app reflects the real-time status.
Budget Breakdown
| Item | Quantity | Unit Cost (LKR) | Total (LKR) |
|---|---|---|---|
| ESP32 Board | 1 | 1,000 | 1,000 |
| NodeMCU Development Board | 4 | 600 | 2,400 |
| IMU Sensor | 1 | 1,000 | 1,000 |
| IR Sensors | 3 | 300 | 900 |
| Battery Pack | 1 | 200 | 200 |
| Plug Sockets | 1 | 1,000 | 1,000 |
| Electronic Door Lock | 1 | 2,500 | 2,500 |
| 230V to 5V Converters | 4 | 300 | 1,200 |
| Relays, Triacs, Resistors, etc. | 4 | 300 | 1,200 |
| Wires, Soldering Components | 1 | 3,000 | 3,000 |
| Other Expenses | 1 | 2,000 | 2,000 |
| Flexible PCB for Wearable Band (with Circuit Components) | 1 | 10,000 | 10,000 |
| Total | 26,400 | ||
Conclusion
This design ensures a seamless, low-latency, and secure gesture-controlled home automation experience. By integrating ESP32 NodeMCU, fingerprint scanner, smart sockets, and MQTT-based communication, our system enables effortless automation and real-time device control.
Future Developments
- Integration with voice assistants like Alexa and Google Assistant.
- Expansion to support more devices and appliances.
- Enhanced machine learning models for improved gesture recognition accuracy.
Commercialization Plans
- Partnering with smart home device manufacturers for integration.
- Launching a subscription-based service for advanced features.
- Expanding to international markets with localized support.
GitHub Repository
Explore our project on GitHub: FlickNest GitHub Repo