How to Label Image in Android using Firebase ML Kit?
Last Updated :
24 Feb, 2021
We have seen many apps in Android in which we will detect the object present in the image whether it may be any object. In this article, we will take a look at the implementation of image labeling in Android using Firebase ML Kit.
What we are going to build in this article?
We will be building a simple application in which we will be capturing an image of any object and from that, we will detect the objects present inside our image with the accuracy level. A sample video is given below to get an idea about what we are going to do in this article. Note that we are going to implement this project using the Java language.
Step by Step Implementation
Step 1: Create a New Project
To create a new project in Android Studio please refer to How to Create/Start a New Project in Android Studio. Note that select Java as the programming language.
Step 2: Connect your app to Firebase
After creating a new project in Android Studio connect your app to Firebase. For connecting your app to firebase. Navigate to Tools on the top bar. After that click on Firebase. A new window will open on the right side. Inside that window click on Firebase ML and then click on Use Firebase ML kit in Android. You can see the option below screenshot.
After clicking on this option on the next screen click on Connect to Firebase option to connect your app to Firebase.
Step 3: Adding dependency for language translation to build.gradle file
Navigate to the Gradle Scripts > build.gradle(Module:app) and add the below dependency in the dependencies section.
// dependency for firebase core.
implementation'com.google.firebase:firebase-core:15.0.2'
// Firebase ML dependency
implementation 'com.google.firebase:firebase-ml-vision:24.0.3'
implementation 'com.google.firebase:firebase-ml-vision-image-label-model:20.0.1'
Step 4: Adding permissions to access the Internet and meta-data in your Android Apps AndroidManifest file
Navigate to the app > AndroidManifest.xml file and add the below code to it. Comments are added in the code to get to know in more detail.
XML
<!-- below line is use to add camera feature in our app -->
<uses-feature
android:name="android.hardware.camera"
android:required="true" />
<!-- permission for internet -->
<uses-permission android:name="android.permission.INTERNET" />
Add the below line inside your application tag.
XML
<meta-data
android:name="com.google.firebase.ml.vision.DEPENDENCIES"
android:value="label" />
Below is the complete code for the AndroidManifest.xml file.
XML
<?xml version="1.0" encoding="utf-8"?>
<manifest xmlns:android="http://schemas.android.com/apk/res/android"
package="com.example.labelimage">
<!-- below line is use to add camera feature in our app -->
<uses-feature
android:name="android.hardware.camera"
android:required="true" />
<!-- permission for internet -->
<uses-permission android:name="android.permission.INTERNET" />
<application
android:allowBackup="true"
android:icon="@mipmap/ic_launcher"
android:label="@string/app_name"
android:roundIcon="@mipmap/ic_launcher_round"
android:supportsRtl="true"
android:theme="@style/Theme.LabelImage">
<activity android:name=".MainActivity">
<intent-filter>
<action android:name="android.intent.action.MAIN" />
<category android:name="android.intent.category.LAUNCHER" />
</intent-filter>
</activity>
<meta-data
android:name="com.google.firebase.ml.vision.DEPENDENCIES"
android:value="label" />
</application>
</manifest>
Step 5: Working with the activity_main.xml file
Navigate to the app > res > layout > activity_main.xml and add the below code to that file. Below is the code for the activity_main.xml file.
XML
<?xml version="1.0" encoding="utf-8"?>
<RelativeLayout
xmlns:android="http://schemas.android.com/apk/res/android"
xmlns:tools="http://schemas.android.com/tools"
android:layout_width="match_parent"
android:layout_height="match_parent"
tools:context=".MainActivity">
<!--image view for displaying our image-->
<ImageView
android:id="@+id/image"
android:layout_width="match_parent"
android:layout_height="300dp"
android:layout_alignParentTop="true"
android:layout_centerHorizontal="true"
android:layout_marginTop="10dp"
android:scaleType="centerCrop" />
<LinearLayout
android:id="@+id/idLLButtons"
android:layout_width="match_parent"
android:layout_height="wrap_content"
android:layout_below="@id/image"
android:orientation="horizontal">
<!--button for capturing our image-->
<Button
android:id="@+id/snapbtn"
android:layout_width="0dp"
android:layout_height="wrap_content"
android:layout_margin="5dp"
android:layout_marginTop="30dp"
android:layout_weight="1"
android:text="SNAP"
android:textSize="25dp"
android:textStyle="bold" />
<!--button for detecting the objects-->
<Button
android:id="@+id/labelBtn"
android:layout_width="0dp"
android:layout_height="wrap_content"
android:layout_margin="5dp"
android:layout_marginTop="30dp"
android:layout_weight="1"
android:text="Label"
android:textSize="25dp"
android:textStyle="bold" />
</LinearLayout>
<!--recycler view for displaying the list of objects-->
<androidx.recyclerview.widget.RecyclerView
android:id="@+id/idRVResults"
android:layout_width="match_parent"
android:layout_height="wrap_content"
android:layout_below="@id/idLLButtons" />
</RelativeLayout>
Step 6: Creating a modal class for storing our data
Navigate to the app > java > your app's package name > Right-click on it > New > Java class and name your class as DataModal and add the below code to it. Comments are added inside the code to understand the code in more detail.
Java
public class DataModal {
// variables for our
// string and confidence.
private String result;
private float confidence;
// constructor
public DataModal(String result, float confidence) {
this.result = result;
this.confidence = confidence;
}
// getter and setter methods
public float getConfidence() {
return confidence;
}
public void setConfidence(float confidence) {
this.confidence = confidence;
}
public String getResult() {
return result;
}
public void setResult(String result) {
this.result = result;
}
}
Step 7: Creating a layout file for displaying our recycler view items
Navigate to the app > res > layout > Right-click on it > New > layout resource file and name it as result_rv_item and add below code to it.
XML
<?xml version="1.0" encoding="utf-8"?>
<androidx.cardview.widget.CardView
xmlns:android="http://schemas.android.com/apk/res/android"
xmlns:app="http://schemas.android.com/apk/res-auto"
android:layout_width="match_parent"
android:layout_height="wrap_content"
android:layout_margin="5dp"
android:elevation="8dp"
app:cardCornerRadius="8dp">
<RelativeLayout
android:layout_width="match_parent"
android:layout_height="wrap_content"
android:layout_margin="3dp">
<!--text view for our result-->
<TextView
android:id="@+id/idTVResult"
android:layout_width="match_parent"
android:layout_height="wrap_content"
android:padding="3dp"
android:text="Result"
android:textColor="@color/black" />
<!--text view for our confidence-->
<TextView
android:id="@+id/idTVConfidence"
android:layout_width="match_parent"
android:layout_height="wrap_content"
android:layout_below="@id/idTVResult"
android:padding="3dp"
android:text="Confidence"
android:textColor="@color/black" />
</RelativeLayout>
</androidx.cardview.widget.CardView>
Step 8: Creating an adapter class for our RecyclerView
Navigate to the app > java > your app's package name > Right-click on it > New > Java class and name it as resultRVAdapter and add the below code to it. Comments are added inside the code to understand the code in more detail.
Java
import android.content.Context;
import android.view.LayoutInflater;
import android.view.View;
import android.view.ViewGroup;
import android.widget.TextView;
import androidx.annotation.NonNull;
import androidx.recyclerview.widget.RecyclerView;
import java.util.ArrayList;
public class resultRVAdapter extends RecyclerView.Adapter<resultRVAdapter.ViewHolder> {
// arraylist for storing our data and context
private ArrayList<DataModal> dataModalArrayList;
private Context context;
// constructor for our variables
public resultRVAdapter(ArrayList<DataModal> dataModalArrayList, Context context) {
this.dataModalArrayList = dataModalArrayList;
this.context = context;
}
@NonNull
@Override
public ViewHolder onCreateViewHolder(@NonNull ViewGroup parent, int viewType) {
// inside on create view holder method we are inflating our layout file which we created.
View view = LayoutInflater.from(parent.getContext()).inflate(R.layout.result_rv_item, parent, false);
return new ViewHolder(view);
}
@Override
public void onBindViewHolder(@NonNull ViewHolder holder, int position) {
// inside on bind view holder method we are setting
// data to each item of recycler view.
DataModal modal = dataModalArrayList.get(position);
holder.resultTV.setText(modal.getResult());
holder.confidenceTV.setText("" + modal.getConfidence());
}
@Override
public int getItemCount() {
// returning the size of array list.
return dataModalArrayList.size();
}
public class ViewHolder extends RecyclerView.ViewHolder {
// creating variables for our text view.
private TextView resultTV, confidenceTV;
public ViewHolder(@NonNull View itemView) {
super(itemView);
// initializing our views with their ids.
resultTV = itemView.findViewById(R.id.idTVResult);
confidenceTV = itemView.findViewById(R.id.idTVConfidence);
}
}
}
Step 9: Working with the MainActivity.java file
Go to the MainActivity.java file and refer to the following code. Below is the code for the MainActivity.java file. Comments are added inside the code to understand the code in more detail.
Java
import android.content.Intent;
import android.graphics.Bitmap;
import android.os.Bundle;
import android.provider.MediaStore;
import android.view.View;
import android.widget.Button;
import android.widget.ImageView;
import android.widget.Toast;
import androidx.annotation.NonNull;
import androidx.appcompat.app.AppCompatActivity;
import androidx.recyclerview.widget.LinearLayoutManager;
import androidx.recyclerview.widget.RecyclerView;
import com.google.android.gms.tasks.OnFailureListener;
import com.google.android.gms.tasks.OnSuccessListener;
import com.google.firebase.ml.vision.FirebaseVision;
import com.google.firebase.ml.vision.common.FirebaseVisionImage;
import com.google.firebase.ml.vision.label.FirebaseVisionImageLabel;
import com.google.firebase.ml.vision.label.FirebaseVisionImageLabeler;
import java.util.ArrayList;
import java.util.List;
public class MainActivity extends AppCompatActivity {
// variables for our image view, image bitmap,
// buttons, recycler view, adapter and array list.
private ImageView img;
private Button snap, labelBtn;
private Bitmap imageBitmap;
private RecyclerView resultRV;
private resultRVAdapter resultRvAdapter;
private ArrayList<DataModal> dataModalArrayList;
@Override
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.activity_main);
// initializing all our variables for views
img = (ImageView) findViewById(R.id.image);
snap = (Button) findViewById(R.id.snapbtn);
labelBtn = findViewById(R.id.labelBtn);
resultRV = findViewById(R.id.idRVResults);
// initializing our array list
dataModalArrayList = new ArrayList<>();
// initializing our adapter class.
resultRvAdapter = new resultRVAdapter(dataModalArrayList, MainActivity.this);
// layout manager for our recycler view.
LinearLayoutManager manager = new LinearLayoutManager(MainActivity.this);
// on below line we are setting layout manager
// and adapter to our recycler view.
resultRV.setLayoutManager(manager);
resultRV.setAdapter(resultRvAdapter);
// adding on click listener for our label button.
labelBtn.setOnClickListener(new View.OnClickListener() {
@Override
public void onClick(View v) {
// calling method
// to label images.
labelImage();
}
});
// adding on click listener for our snap button.
snap.setOnClickListener(new View.OnClickListener() {
@Override
public void onClick(View v) {
// calling a method
// to capture an image.
dispatchTakePictureIntent();
}
});
}
static final int REQUEST_IMAGE_CAPTURE = 1;
private void dispatchTakePictureIntent() {
// inside this method we are calling an implicit intent to capture an image.
Intent takePictureIntent = new Intent(MediaStore.ACTION_IMAGE_CAPTURE);
if (takePictureIntent.resolveActivity(getPackageManager()) != null) {
// calling a start activity for result when image is captured.
startActivityForResult(takePictureIntent, REQUEST_IMAGE_CAPTURE);
}
}
@Override
protected void onActivityResult(int requestCode, int resultCode, Intent data) {
super.onActivityResult(requestCode, resultCode, data);
// inside on activity result method we are setting
// our image to our image view from bitmap.
if (requestCode == REQUEST_IMAGE_CAPTURE && resultCode == RESULT_OK) {
Bundle extras = data.getExtras();
imageBitmap = (Bitmap) extras.get("data");
// on below line we are setting our
// bitmap to our image view.
img.setImageBitmap(imageBitmap);
}
}
private void labelImage() {
// inside the label image method we are calling a
// firebase vision image and passing our image bitmap to it.
FirebaseVisionImage image = FirebaseVisionImage.fromBitmap(imageBitmap);
// on below line we are creating a labeler for our image bitmap
// and creating a variable for our firebase vision image labeler.
FirebaseVisionImageLabeler labeler = FirebaseVision.getInstance().getOnDeviceImageLabeler();
// calling a method to process an image and adding on success listener method to it.
labeler.processImage(image).addOnSuccessListener(new OnSuccessListener<List<FirebaseVisionImageLabel>>() {
@Override
public void onSuccess(List<FirebaseVisionImageLabel> firebaseVisionImageLabels) {
// inside on success method we are running a loop to get the data from our list.
for (FirebaseVisionImageLabel label : firebaseVisionImageLabels) {
// on below line we are getting text from our list.
String text = label.getText();
// on below line we are getting its entity id
String entityId = label.getEntityId();
// on below line we are getting the
// confidence level of our modal.
float confidence = label.getConfidence();
// after getting all data we are passing it to our array list.
dataModalArrayList.add(new DataModal(text, confidence));
// after adding a new data we are notifying
// our adapter that data has been updated.
resultRvAdapter.notifyDataSetChanged();
}
}
}).addOnFailureListener(new OnFailureListener() {
@Override
public void onFailure(@NonNull Exception e) {
// error handling for on failure method
Toast.makeText(MainActivity.this, "Fail to get data..", Toast.LENGTH_SHORT).show();
}
});
}
}
Now run your app and see the output of the app.
Output:
Note: If you are facing the following NDK at ~/Library/Android/sdk/ndk-bundle did not have a source.properties file error then please refer to this article.
Similar Reads
Android Architecture Android architecture contains a different number of components to support any Android device's needs. Android software contains an open-source Linux Kernel having a collection of a number of C/C++ libraries which are exposed through application framework services. Among all the components Linux Kern
5 min read
Types of Functional dependencies in DBMS In relational database management, functional dependency is a concept that specifies the relationship between two sets of attributes where one attribute determines the value of another attribute. It is denoted as X â Y, where the attribute set on the left side of the arrow, X is called Determinant,
6 min read
Understanding TF-IDF (Term Frequency-Inverse Document Frequency) TF-IDF (Term Frequency-Inverse Document Frequency) is a statistical measure used in natural language processing and information retrieval to evaluate the importance of a word in a document relative to a collection of documents (corpus). Unlike simple word frequency, TF-IDF balances common and rare w
6 min read
List Comprehension in Python List comprehension is a way to create lists using a concise syntax. It allows us to generate a new list by applying an expression to each item in an existing iterable (such as a list or range). This helps us to write cleaner, more readable code compared to traditional looping techniques.For example,
4 min read
Android Tutorial In this Android Tutorial, we cover both basic and advanced concepts. So whether you are a fresher (graduate) or an experienced candidate with several years of Android Development experience, you can follow this Android tutorial to kick-start your journey in Android app development. Our Android Tutor
15+ min read
Mean, Median and Mode Mean, Median, and Mode are measures of the central tendency. These values are used to define the various parameters of the given data set. The measure of central tendency (Mean, Median, and Mode) gives useful insights about the data studied, these are used to study any type of data such as the avera
15+ min read
Stock Buy and Sell - Max one Transaction Allowed Given an array prices[] of length N, representing the prices of the stocks on different days, the task is to find the maximum profit possible by buying and selling the stocks on different days when at most one transaction is allowed. Here one transaction means 1 buy + 1 Sell.Note: Stock must be boug
8 min read
30 Days of SQL - From Basic to Advanced Level This basic to advanced SQL tutorial covers the entire SQL syllabus in a structured way and provides the best learning material and strategies to master complete SQL in 30 Days. We have laid out the complete SQL roadmap, and following this roadmap, you will learn all the concepts of SQL. All Importan
8 min read
Data Scientist Roadmap - A Complete Guide [2025] Welcome to your comprehensive Data Science Roadmap! If youâve ever wondered, about âSteps or Path to Become a Data Scientistâ, youâre in the right place. This guide is perfect for Data Science for Beginners and seasoned professionals alike, covering everything from mastering Python for Data Science
8 min read
What is An Event Loop in JavaScript? The event loop is an important concept in JavaScript that enables asynchronous programming by handling tasks efficiently. Since JavaScript is single-threaded, it uses the event loop to manage the execution of multiple tasks without blocking the main thread.JavaScriptconsole.log("Start"); setTimeout(
4 min read