Make Your Android App Smarter

综合技术 2017-05-25 阅读原文

Artificial Intelligence and its branches such as machine learning quickly gaining in popularity with both users and developers alike. While there is no single recipe to creating the next greatest app that everyone will use, but I can guarantee that one of the ingredients in the usage of thoughtfully incorporated AI.

But creating your own AI solution can be very complex and tricky. The good news is, there are already some very powerful and easy-to-use solutions provided by companies such as Google, Amazon, and Microsoft. These comanies offer services and libraries with popular features such as Face Recognition, OCR (Optical Character Recognition), Classification Model, and so on. (References and links can be found at the end of this post).

One solution that we had chance to use here at Avenue Code is the Google Mobile Vision API, which is part of the Google Play service in the Android platform. Some features of this library are not exclusive to Android. For instance, iOS developers can also use the Face Recognition API - check out the documentation! This post is dedicated to how we can use Face Recognition in our app with Google Mobile Vision API.

Creating the next MSQRD or Snapchat is not rocket science with MV API. To quote the API description: “ It's designed to better detect human faces in images and video for easier editing. It's smart enough to detect faces even at different orientations -- so if your subject's head is turned sideways, it can detect it. Specific landmarks can also be detected on faces, such as the eyes, the nose, and the edges of the lips.

So let’s get started! First of all, create a project in Android Studio. In order to use the Google Mobile Vision API your gradle file from the application level should import this dependency. At the time of this post, the current version of play-service-vision is 10.2.4. You can check out the documentation here:

compile ''

Your Android Manifest file should enable the service. Place this below code in the Application tag.

Our app is based on the Google sample code for the Camera2 API. To focus on how we can use the Face Recognition API, we will abstract the camera layer and go directly to the heart of what this post is all about. If you want more details on how the camera2 API works, please take a look at the github repository:

The architecture of our application consists of two packages: camera and face_recognition. The camera package is entirely based on camera2basic repository, with the only extra method being the startFaceRecognition in FragmentCameraPreview:

private void startFaceRecognition() { Bundle bundle = new Bundle(); bundle.putSerializable("image", mSavedFile); FaceRecognitionFragment frResultFragment = new FaceRecognitionFragment(); frResultFragment.setArguments(bundle); getFragmentManager().beginTransaction().replace(, frResultFragment).commit(); }

This method only retrieves the photo that user has taken. The result of the recognition lays on the FaceRecogntionFragment. Before we can actually call the API, we need to prepare our ImageView to display the processed result. We are writing on ImageView itself so we need Bitmap, Paint and Canvas for this to work

BitmapFactory.Options options = new BitmapFactory.Options(); options.inMutable = true; Bitmap bitmap = BitmapFactory.decodeFile(file.getAbsolutePath(), options);

We need to make the Bitmap mutable so we can write on it later - otherwise we will end up with an exception like this: java.lang.IllegalStateException: Immutable bitmap passed to Canvas constructor

Then do the little red circles which shows parts of the face:

Paint rectPaint = new Paint(); rectPaint.setStrokeWidth(5); rectPaint.setColor(Color.RED); rectPaint.setStyle(Paint.Style.STROKE);

And finally, create the canvas where these red circles will be drawn:

Canvas canvas = new Canvas(imageBitmap); canvas.drawBitmap(imageBitmap, 0, 0, null);

Face Recognition Coding

Up to this point, nothing magical is happening. We're only warming up the stage for the big player: FaceDetector.

Beware when importing this dependency: there are two FaceDetectors, one from which we are NOT interested in, and the one from android package. This is our guy.

FaceDetector faceDetector = new FaceDetector.Builder(getActivity()) .setTrackingEnabled(false) .setLandmarkType(FaceDetector.ALL_LANDMARKS) .build();

First we need to setTrackingEnabled to false to get better performance and accuracy for single image processing. We enable this property for live video, for example.

Next we need to check if the service of Face Detector is operational:

if (!faceDetector.isOperational()) { new AlertDialog.Builder(getActivity()).setMessage("Could not set up the face detector!").show(); return; }

Sometimes it takes a little while to download this dependency.

Now let’s detect faces! Create a frame using the Bitmap, then call the detect method on the FaceDetector, using this frame, to get back a SparseArray of Face objects:

Frame frame = new Frame.Builder().setBitmap(imageBitmap).build(); SparseArray faces = faceDetector.detect(frame);

Little Challenge

To make this app funnier let’s recognize both eyes and replace with the Avenue Code Big ‘A’ icon, like this:

For this, create a Bitmap the references this resource:

Bitmap acBitmap = BitmapFactory.decodeResource(getResources(),;

It’s very easy to retrieve the coordinates for right and left eye. Just loop through the sparseArray and get the landmark of each face like this:

for (int i = 0; i < faces.size(); i++) {

Face face = faces.valueAt(i);

for (Landmark landmark : face.getLandmarks()) {

int cx = (int) (landmark.getPosition().x);

int cy = (int) (landmark.getPosition().y);

float radius = 10.0f;

if(landmark.getType() == Landmark.LEFT_EYE || landmark.getType() == Landmark.RIGHT_EYE) {

canvas.drawBitmap(acBitmap, cx - 100, cy - 100, null);

} else {

canvas.drawCircle(cx, cy, radius, rectPaint); }



And finally just set the result Bitmap:

mImageView.setImageDrawable(new BitmapDrawable(getResources(), imageBitmap));


That’s it. Easy and painless. With less than a line of code, we have applied an AI in our app. You can checkout this project in github:

This is a tiny little glimpse into what you can do with the Google Mobile Vision API - there's plenty more to learn and play with! Hoping this was fun and enjoyable - questions? Feedback? Leave them in the comments!


Google Mobile Vision API:

Google AI Cloud Solution:



Camera2 Sample Repository:

Google Code Lab:

责编内容by:Avenue Code Snippets 【阅读原文】。感谢您的支持!


Android启动方式简述 前言 我们点击Android应用图标启动应用程序的时候,有时候可以马上启动,但有时候很缓慢并进入空白页,这是为什么呢?这是因为Android的启动方式分为冷启动和热启动。 Android启动方式 Android启动方式有两种:冷启动,热启动 ...
Working with SMS in Android, Read Messages from In... Permissions Add these two permissions in the Android ManifestRead SMS from Inbox Reading SMS from inbox is not a big task, but your user n...
Dude, Where’s (the Instrumentation for) My Car? My car isn’t what anyone would refer to as “smart.” It’s an eleven-year old Toyota Avensis, and with no hubcaps, mismatched side mirrors, and “wash me...
Android魔法(第三弹)—— 一步步实现对折页面... Android魔法系列: 项目的github地址: FastWidget4Android 很多炫酷的自定义效果,欢迎fork和star! 本篇文章主要去实现一个对折页面的效果,主...
Android手势处理看这一篇就够了 前言Android提供了几种处理手势的监听器,但是监听器之间的关系比较乱,新手不太容易知道该用哪一种?什么时候该重写onTouchEvent什么时候直接实现监听器。本文对Android提供的手势监听器一一说明,并在文末给出实际项目的方案选择建议。 常见手势有:单击,长按,双击,滑...