Make Your Android App Smarter

综合技术 2017-05-25 阅读原文

Artificial Intelligence and its branches such as machine learning quickly gaining in popularity with both users and developers alike. While there is no single recipe to creating the next greatest app that everyone will use, but I can guarantee that one of the ingredients in the usage of thoughtfully incorporated AI.

But creating your own AI solution can be very complex and tricky. The good news is, there are already some very powerful and easy-to-use solutions provided by companies such as Google, Amazon, and Microsoft. These comanies offer services and libraries with popular features such as Face Recognition, OCR (Optical Character Recognition), Classification Model, and so on. (References and links can be found at the end of this post).

One solution that we had chance to use here at Avenue Code is the Google Mobile Vision API, which is part of the Google Play service in the Android platform. Some features of this library are not exclusive to Android. For instance, iOS developers can also use the Face Recognition API - check out the documentation! This post is dedicated to how we can use Face Recognition in our app with Google Mobile Vision API.

Creating the next MSQRD or Snapchat is not rocket science with MV API. To quote the API description: “ It's designed to better detect human faces in images and video for easier editing. It's smart enough to detect faces even at different orientations -- so if your subject's head is turned sideways, it can detect it. Specific landmarks can also be detected on faces, such as the eyes, the nose, and the edges of the lips.

So let’s get started! First of all, create a project in Android Studio. In order to use the Google Mobile Vision API your gradle file from the application level should import this dependency. At the time of this post, the current version of play-service-vision is 10.2.4. You can check out the documentation here:

compile ''

Your Android Manifest file should enable the service. Place this below code in the Application tag.

Our app is based on the Google sample code for the Camera2 API. To focus on how we can use the Face Recognition API, we will abstract the camera layer and go directly to the heart of what this post is all about. If you want more details on how the camera2 API works, please take a look at the github repository:

The architecture of our application consists of two packages: camera and face_recognition. The camera package is entirely based on camera2basic repository, with the only extra method being the startFaceRecognition in FragmentCameraPreview:

private void startFaceRecognition() { Bundle bundle = new Bundle(); bundle.putSerializable("image", mSavedFile); FaceRecognitionFragment frResultFragment = new FaceRecognitionFragment(); frResultFragment.setArguments(bundle); getFragmentManager().beginTransaction().replace(, frResultFragment).commit(); }

This method only retrieves the photo that user has taken. The result of the recognition lays on the FaceRecogntionFragment. Before we can actually call the API, we need to prepare our ImageView to display the processed result. We are writing on ImageView itself so we need Bitmap, Paint and Canvas for this to work

BitmapFactory.Options options = new BitmapFactory.Options(); options.inMutable = true; Bitmap bitmap = BitmapFactory.decodeFile(file.getAbsolutePath(), options);

We need to make the Bitmap mutable so we can write on it later - otherwise we will end up with an exception like this: java.lang.IllegalStateException: Immutable bitmap passed to Canvas constructor

Then do the little red circles which shows parts of the face:

Paint rectPaint = new Paint(); rectPaint.setStrokeWidth(5); rectPaint.setColor(Color.RED); rectPaint.setStyle(Paint.Style.STROKE);

And finally, create the canvas where these red circles will be drawn:

Canvas canvas = new Canvas(imageBitmap); canvas.drawBitmap(imageBitmap, 0, 0, null);

Face Recognition Coding

Up to this point, nothing magical is happening. We're only warming up the stage for the big player: FaceDetector.

Beware when importing this dependency: there are two FaceDetectors, one from which we are NOT interested in, and the one from android package. This is our guy.

FaceDetector faceDetector = new FaceDetector.Builder(getActivity()) .setTrackingEnabled(false) .setLandmarkType(FaceDetector.ALL_LANDMARKS) .build();

First we need to setTrackingEnabled to false to get better performance and accuracy for single image processing. We enable this property for live video, for example.

Next we need to check if the service of Face Detector is operational:

if (!faceDetector.isOperational()) { new AlertDialog.Builder(getActivity()).setMessage("Could not set up the face detector!").show(); return; }

Sometimes it takes a little while to download this dependency.

Now let’s detect faces! Create a frame using the Bitmap, then call the detect method on the FaceDetector, using this frame, to get back a SparseArray of Face objects:

Frame frame = new Frame.Builder().setBitmap(imageBitmap).build(); SparseArray faces = faceDetector.detect(frame);

Little Challenge

To make this app funnier let’s recognize both eyes and replace with the Avenue Code Big ‘A’ icon, like this:

For this, create a Bitmap the references this resource:

Bitmap acBitmap = BitmapFactory.decodeResource(getResources(),;

It’s very easy to retrieve the coordinates for right and left eye. Just loop through the sparseArray and get the landmark of each face like this:

for (int i = 0; i < faces.size(); i++) {

Face face = faces.valueAt(i);

for (Landmark landmark : face.getLandmarks()) {

int cx = (int) (landmark.getPosition().x);

int cy = (int) (landmark.getPosition().y);

float radius = 10.0f;

if(landmark.getType() == Landmark.LEFT_EYE || landmark.getType() == Landmark.RIGHT_EYE) {

canvas.drawBitmap(acBitmap, cx - 100, cy - 100, null);

} else {

canvas.drawCircle(cx, cy, radius, rectPaint); }



And finally just set the result Bitmap:

mImageView.setImageDrawable(new BitmapDrawable(getResources(), imageBitmap));


That’s it. Easy and painless. With less than a line of code, we have applied an AI in our app. You can checkout this project in github:

This is a tiny little glimpse into what you can do with the Google Mobile Vision API - there's plenty more to learn and play with! Hoping this was fun and enjoyable - questions? Feedback? Leave them in the comments!


Google Mobile Vision API:

Google AI Cloud Solution:



Camera2 Sample Repository:

Google Code Lab:

责编内容by:Avenue Code Snippets 【阅读原文】。感谢您的支持!


Android破解学习之路(十)—— 我们恋爱吧 三色绘恋 二次破解... 前言 好久没有写破解教程了(我不会告诉你我太懒了),找到一款恋爱游戏,像我这样的宅男只能玩玩恋爱游戏感觉一下恋爱的心动了。。 这款游戏免费试玩,但是后续章节得花6元钱购买,我怎么会有钱呢,而且身在吾爱的大家庭里,不破解一波怎么对得起...
BubbleGum BubbleGum - Sweet Android Gradients BubbleGum was designed to quickly apply gradients into different UI elements o...
把脑子扔掉,让Gradle帮你实现Android组件化模块构建... 随着App的不断迭代,业务会变得越来越复杂,业务模块会越来越多,且每个模块的代码也会变得越来越多。为了应对这一场景,我们需要把不同的业务模块划分成一个个组件,在修改业务代码的时候只需要在对应模块修改就可以了。通过高内聚,低耦合的业务模块来保...
Android(Linux) 输入子系统解析 Android 源码分析系列综述博文: Android 系统源码分析综述 前言 Android/Linux 输入设备总类繁杂,常见的有按键、键盘、触摸屏、鼠标、摇杆等,之前其驱动都是采用字符设备、misc 设备处理的,但是如此多...
Rxjava这一篇就够了,墙裂推荐 最近项目里面有用到Rxjava框架,感觉很强大的巨作,所以在网上搜了很多相关文章,发现一片文章很不错,今天把这篇文章分享给大家,感谢原作者的倾心贡献,那么下面大家来看看这篇文章: 前言 我从去年开始使用 RxJava ,到现在一年多...