For Developers

Implementation details and decisions. How to extend the existing system

Table of Contents

Notes on the Implementation

Implementation details and decisions

We wanted Smart Mirror to be as modular possible and our microservice based architecture helped us accomplish that. Except for the doctor app, the modules can be run independently. The emotion detection is done in a way that could be easily implemented in other systems other than Smart Mirror.

While we envisioned SmartMirror to be implemented on an actual mirror, due to the lack of hardware, we decided for a fully digital implementation, as proof of concept.

Due to the nature of the project, SmartMirror captures both sound and image. Since this is being deployed in a remote machine, we had to find a way to access the user microphone and camera.
To do so, we relied on HTLM5 video tag, to access the webcam and Javascript Speech Recognition library, for voice commands, effectively shifting the capturing responsibilities to the client-side browser. The downsides of this are that the Speech Recognition features only work on Chrome Browsers (at the time) and that the application can only be accessed through HTTPS.


How it was implemented

SmartMirror consists of 2 main modules plus a service for proof of concept of the usage of the system by third-parties:

Component Technology Runtime Purpose
Core Java
Spring Framework 		Running on a docker container, with the name esp30-smartmirror, on port 30043 of 192.168.160.103. This module exposes the mirror webpage. On this page the user can take a picture, either by clicking or by voice commands. The picture is then converted to base64 on the javascript and sent via a POST request to the backend to be redirected to the emotion detection module to be processed. This module also exposes a REST API. This can be used by the doctor to check on his patients.
Emotion Detection Module Python
OpenCV library 		Running on a docker container, with the name esp30-smartmirror-emotiondetection, on 192.168.160.103. This module receives images in Base64 via messages on a Kafka topic, and processes them. If the module can classify the emotion of the image, it replies with a JSON file containing the resulting emotion.
Doctor application Java
Spring Framework 		Running on a docker container, with the name esp30-smartmirror-docapp, on port 30020 of 192.168.160.103. This module uses our API and shows the information it fetches in Web Page.

How to extend the existing system

Quick tutorial

To start extending SmartMirror, you should first familiarize yourself with the repository structure:
  • On the folder “/src” you can find the code, test and resources of the SmartMirror core. The pom.xml relating to this module is on the root of the project
  • The emotion detection module source code and corresponding models are available on the folder “/python_src”
  • As for the Doctor App, you can find the code, resources and pom.xml on the /api_microservice/api-microservice-smartMirror folder.
Have also in mind that:
  • Each of the folders mentioned above contains a Dockerfile to build the corresponding docker image.
  • Both Spring Boot apps run on the embedded Apache Tomcat server, so no additional application service (e.g Payara) is necessary but can be used if wanted by deploying the JAR file that results from running mvn deploy -s settings.xml.
  • The SmartMirror core module front-end only works via HTTPS, which means the connection as to be secured with a SSL certificate. This certificate is used is kept on the resources of the module and has a password. Also due to HTTPS, the Apache Tomcat server port is configured on 8443.
  • The SmartMirror core app assumes there’s a MySQL database running on port 3307, with the credentials (root,password). This can be changed at any time on application.properties file.
  • The SmartMirror core app tries to init a connection with a Kafka broker on port 9093 after accessing the Mirror page. This configuration is on the WebCamController and can/should be changed accordingly to your needs.
  • The topic p30-classification needs to be created previously on the Kafka broker before deploying the application, or the system will not work.

arrow_upward Return to Top