Open In App

How to Perform Sound Classification with ml5.js?

Last Updated : 31 Jul, 2024
Comments
Improve
Suggest changes
Like Article
Like
Report

ml5.js is a JavaScript library that makes machine learning easy and accessible to use in web applications. This is a beginner-friendly library that provides an API that you can include in your project to use pre-trained machine-learning models in web projects. So, even if you are a beginner to machine-learning, and want to integrate machine-learning models into their web application ml5.js is for you.

We will look at how to perform sound classification using ml5.js. To be precise enough, we would be using the SpeechCommands18w model, which is a pre-trained model for sound classification of 18 words available provided by ml5.js. Additionally, we would also use p5.js, which is another JavaScript library that complements ml5.js to write code to perform sound classification.

Approach

Let's start by including ml5.js and p5.js libraries in your HTML file inside the head tag. Also, include a link of the JavaScript file (script.js) to your HTML file inside the body tag, this is where you will be writing your JavaScript code. This is called external linking of JavaScript files, however, you can also include JavaScript directly within the HTML file. we would be using the SpeechCommands18w model. The SpeechCommands18w model is a pre-trained model made available by ml5.js to classify sounds of 18 words, namely, one, two, three, four, five, six, seven, eight, nine, up, down, left, right, go, stop, yes, and no.

Example: This example shows the implementation of the above-explained approach.

HTML
<!DOCTYPE html>
<html lang="en">

<head>
    <meta charset="UTF-8" />
    <meta name="viewport" 
          content="width=device-width,
                   initial-scale=1.0" />
    <title>Sound classification</title>
    <script src=
"https://cdnjs.cloudflare.com/ajax/libs/p5.js/1.9.4/p5.min.js"></script>
    <script src=
"https://unpkg.com/[email protected]/dist/ml5.min.js"></script>
</head>

<body>
    <script src="./script.js"></script>
</body>

</html>
JavaScript
// To store sound classifier object
let classify;

// This varible is to keep track of 
// the word that model predicts
let predictedWord = "";

// This array consists of words that
// the SpeechCommand18w model can predict.
let words = [
    "zero",
    "three",
    "six",
    "nine",
    "left",
    "stop",
    "one",
    "four",
    "seven",
    "up",
    "right",
    "yes",
    "two",
    "five",
    "eight",
    "down",
    "go",
    "no",
];

/* 
The 'preload' function loads SpeechCommand18w
with a confidence threshold of 0.6 .
This means that the model only accepts 
prediction above this confidence level.
*/
function preload() {
    let options = { probabilityThreshold: 0.6 };
    classify = ml5.soundClassifier("SpeechCommands18w", options);
}

/*
The 'setup' function runs once when program starts.
Here, a canvas is created and classification process starts.
The 'gotresult' function handle predictions and errors. 
*/
function setup() {
    createCanvas(650, 450);
    classify.classifyStart(gotResult);
}

/*
The 'gotResult' function is called
when classifier makes a prediction
Results to the prediction are in an 
array ordered by confidence.
The first label is one the model feels
most confident about, at indicated by result[0].
*/
function gotResult(results) {
    console.log(results);
    predictedWord = results[0].label;
}

/*
This function is to draw a list of
18 words recognized by model in tha canvas.
*/
function displayWords() {
    textAlign(CENTER, CENTER);
    textSize(28);
    fill(0);
    text("Speak one of these words!", width / 2, 40);
    let x = 125;
    let y = 150;
    for (let i = 0; i < words.length; i++) {
        fill(120);
        text(words[i], x, y);
        y += 50;
        if ((i + 1) % 6 === 0) {
            x += 200;
            y = 150;
        }
    }
}

/*
This function runs continuously in a
loop, it updates and renders
the canvas repeatedly.
It shows the predicted word.
*/
function draw() {
    background(249);
    displayWords();
    if (predictedWord !== "") {
        fill("#2f8d46");
        textAlign(CENTER, CENTER);
        textSize(70);
        text(predictedWord, width / 2, 100);
    }
}

Output:


Next Article

Similar Reads