A Machine for Living In

Sensing Strategy:

• Physical Sensors
  • vibration, temperature, humidity, beam break, physical switch, Passive Infra Red (PIR)
• Audio Nodes
  • ICS43432 MEMS microphone interface via I2S
  • recording  24bit 48kHz audio
  • running on-device voice activity detection written in python (webrtcvad)
• Video nodes
  • chip camera
  • on-device motion detection using motion field thresholding from  h264 encoder block in the SOC.
  • v2: on-device image analysis with movidius VPU
  • FLIR camera

Analysis:

• Audio Analysis:
  • feature extraction (12 points)
    • 4 Mel-Frequency Cepstral Coefficients
    • Spectral Centroid
    • 6-point Chromagram
    • Fast Fourier Transform crest
  • clustering
    • k-Means
    • HDBSCAN
• Video Analysis:
  • background removal and motion path estimation (knn, MOG, HOG with opencv)
  • v2: unwarping and Google Image Net classifier
• Sensor Analysis:
  • time distribution of sensor activations over the day