---

Repository for the display side of the Soundlights project.

Soundlights consists on an intelligent acoustic sensor (SENS, see below) connected to a display or "soundlight" that throws messages in real time aiming to influence on the behaviour of citizens in order to improve the acoustic quality of the environment.

About The Project

''Soundlights: Distributed Open Sensors Network and Citizen Science for the Collective Management of the City's Sound Environments'' (9382417), is a collaborative project between the Music Technology Group (Universitat Pompeu Fabra) and Bitlab Cooperativa Cultural.

The Smart Environmental Noise System (SENS) is an advanced sensor technology designed for real-time acoustic monitoring, with a focus on urban environments. Built on a Raspberry Pi platform, SENS captures sound continuously and processes it locally using custom-developed software based on small and efficient artificial intelligence algorithms. SENS calculates acoustic parameters, including Sound Pressure Level (SPL), and makes predictions of the perceptual sound attributes of pleasantness and eventfulness (ISO 12913), along with detecting the presence of specific sound sources such as vehicles, birds, and human activity, among others. To safeguard privacy, all processing occurs directly on the device in real-time ensuring that no audio recordings are permanently stored or transferred. Additionally, the system transmits the extracted audio representation through the wireless network to a remote server or to a display for visualization, made possible using mobile data connectivity. See SENS Github repository for more information.

Project funded by BIT Habitat (Ajuntament de Barcelona) under the program La Ciutat Proactiva; and by the IA y Música: Cátedra en Inteligencia Artificial y Música (TSI-100929-2023-1) by the Secretaría de Estado de Digitalización e Inteligencia Artificial and NextGenerationEU under the program Cátedras ENIA 2022.

(back to top)

Reference

• Amaia Sagasti, Martín Rocamora, Frederic Font: Prediction of Pleasantness and Eventfulness Perceptual Sound Qualities in Urban Soundscapes - DCASE Workshop 2024 Paper link DCASE webpage
• Amaia Sagasti Martínez - MASTER THESIS: Prediction of Pleasantness and Eventfulness Perceptual Sound Qualities in Urban Soundscapes - Sound and Music Computing Master (Music Technology Group, Universitat Pompeu Fabra - Barcelona) Master Thesis Report link Zenodo

Components

This project is implemented in a RaspberryPi model B. Check the following schema to see the components:

Set up

1. Clone this repository

   git clone https://github.com/MTG/sens-display.git
   

2. Install python==3.10 download web.

   Follow instructions below (or link)

   sudo apt-get update
   
   sudo apt-get install build-essential tk-dev libncurses5-dev libncursesw5-dev libreadline6-dev libdb5.3-dev libgdbm-dev libsqlite3-dev libssl-dev libbz2-dev libexpat1-dev liblzma-dev zlib1g-dev libffi-dev
   
   wget https://www.python.org/ftp/python/3.10.14/Python-3.10.14.tar.xz # or download directly from link above
   
   # Navigate to directory where download is
   tar -xvf Python-3.10.14.tar.xz
   
   cd Python-3.10.14
   
   ./configure --enable-optimizations # be patient
   
   make -j 4 # be waaay more patient here
   
   sudo make altinstall
   
   python3.10 --version # to verify installation
   

3. It is recommended to create an environment. For example:

   python -m venv env_name
   

   Then, activate your environment, navigate to your project directory and install the requirements:

   source env_name/bin/activate
   
   cd sens-display
   
   pip install -r requirements.txt
   

4. To use Adafruit's RGB Matrix Bonnet, we need a specific python library. Check page 17 of the datasheet found in this link to see the instructions below in more detail:

   # Download and Installation
   curl https://raw.githubusercontent.com/adafruit/Raspberry-Pi-Installer-Scripts/main/ 
   rgb-matrix.sh >rgb-matrix.sh 
   sudo bash rgb-matrix.sh
   
   # When asked [y/n], type "y"
   
   # When asked about bonnet type, select << Adafruit RGB Matrix Bonnet >>
   
   # When asked about quality, select << convenience >>
   
   # Library rgbmatrix==0.0.1 will be installed
   

   NOTE: Make sure that the installed rgbmatrix library is accessible from the environment you created. You may need to change python paths in rgb-matrix.sh prior to installation process.

5. In sens-display/config.py, there is a section where you must specify the LED matrices configuration you are using. Take into consideration that the chosen RGB bonnet only accepts matrices joint in chain. However, our goal was to have a square screen, therefore, the configuration was set in chain but, when coding, images where located so that they would fit the 64x64 format.

Run code

Currently, Soundlights is prepared to receive two types of messages from SENS:

• When all sound sources, P and E predictions are sent. Simply run:

  python main.py all_sources
  

  The expected input message structure is:

  {"sources":
      {"birds": 0.012,
      "construction": 0.007,
      "dogs": 0.015,
      "human": 0.065,
      "music": 0.493,
      "nature": 0.332,
      "siren": 0.138,
      "vehicles": 0.385},
  "P_inst": 0.052,
  "P_intg": 0.147,
  "E_inst": -0.002,
  "E_intg": -0.199,
  "leq": 77.0,
  "LAeq": 54.1,
  "datetime": "2025-01-31T15:12:05+01:00"}
  

• We carried out an experiment in a library. In this case, SENS was only predicting the presence of human activity. In this case, run:

  python main.py library
  

  The expected input message structure is:

  {"sources":
      {"human": 0.065},
  "leq": 77.0,
  "LAeq": 54.1,
  "datetime": "2025-01-31T15:12:05+01:00",
  "human_threshold": 0.25}
  

For other input messages, you will have to create a function similar to process_message_all() or process_message_library() from utils/data_handler