caffe学习(八)添加模块并JNI移植

添加识别代码

examples文件夹下面新建文件夹porno_recognition,并添加cpp源码。这里从caffe/examples/cpp_classification/classification.cpp中复制了一段识别的代码,并修改classify函数以及添加load函数。由于要生成动态链接库,所以把main函数屏蔽掉。

修改Makefile

找到$(EXAMPLE_BINS): %.bin : %.o | $(DYNAMIC_NAME)部分,添加-fPIC -shared,重新make all,编译生成 build/examples/porno_recognition/porno_recognition.bin,可以看到这个新生成的bin比之前的小,确认其为动态链接库的模型,执行mv porno_recognition.bin libSuningPore.so

JNI

添加java接口函数

创建文件夹com\suning\pore并在其目录下面添加SuningPore.java,代码如下:

package com.suning.pore;

public class SuningPore
{
    static
    {        
        System.loadLibrary("SuningPore");
        System.out.println("***载入libSuningPore.so完成***");
    }

    public native static int load(String ocrDataPath, String libPath, String logPath);
    public native static String start(String imgUrl, byte[] pic);
}

这其中的loadLibrary函数用来加载libSuningPore.so动态链接库。load和start两个函数是这个动态链接库提供的两个接口函数。

创建.h文件

在com的上一级目录下,执行

javah com.suning.pore.SuningPore

生成文件com_suning_pore_SuningPore.h

创建cpp接口实现代码

创建com_suning_pore_SuningPore.cpp,并实现对应头文件中定义的两个接口函数的实现。并将之前实现的porno_recognition.cpp与该文件整合为一个文件

#include "com_suning_pore_SuningPore.h"
#include <string>

int jni_load(const char *model_path, const char* lib_path, const char *log_path);

std::string jni_start(const char* imgUrl, const unsigned char *p_imgdata, int data_size);

JNIEXPORT jint JNICALL Java_com_suning_pore_SuningPore_load
(JNIEnv *env, jclass obj, jstring model_path, jstring lib_path, jstring log_path)
{
    const char *model_string = (env)->GetStringUTFChars(model_path, 0);
    const char *lib_string = (env)->GetStringUTFChars(lib_path, 0);
    const char *log_string = (env)->GetStringUTFChars(log_path, 0);

    jint ret = jni_load(model_string, lib_string, log_string);

    (env)->ReleaseStringUTFChars(model_path, model_string);
    (env)->ReleaseStringUTFChars(log_path, log_string);
    (env)->ReleaseStringUTFChars(lib_path, lib_string);

    return ret;
}

JNIEXPORT jstring JNICALL Java_com_suning_pore_SuningPore_start
(JNIEnv *env, jclass obj, jstring img_url, jbyteArray in_buf)
{
    const char *url_string = (env)->GetStringUTFChars(img_url, 0);

    jbyte *cbuf;
    cbuf = (env)->GetByteArrayElements(in_buf, 0);
    jsize in_len = (env)->GetArrayLength(in_buf);

    const unsigned char *p_imgdata = (unsigned char*)cbuf;
    int data_size = in_len;

    jstring ret = 0;
    if (p_imgdata == NULL || data_size <= 0)
    {
        return ret;
    }

    const std::string out_str = jni_start(p_imgdata, data_size);

    (env)->ReleaseStringUTFChars(img_url, url_string);
    (env)->ReleaseByteArrayElements(in_buf, cbuf, 0);

    if (!out_str.empty()) {
        ret = (env)->NewStringUTF(out_str.c_str());
    }
    return ret;
}

然后将com_suning_pore_SuningPore.cppcom_suning_pore_SuningPore.h放入caffe/examples/porno_recognition文件夹中,修改Makefile.config的中加入java的include,重新编译生成com_suning_pore_SuningPore.bin(注意:该文件实际上是一个动态链接库,不可直接执行)。

### Makefile.config ###
# INCLUDE_DIRS := ...
    /usr/lib/jvm/java/include \
    /usr/lib/jvm/java/include/linux \

添加java测试程序

com/suning/pore下添加TestSuningPore.java并添加如下代码:

package com.suning.pore;

import java.io.File;
import java.io.FileInputStream;
import java.io.FileNotFoundException;

public class TestSuningPore
{
    public static void main(String[] args) throws Exception
    {
        // TODO Auto-generated method stub

        TestSuningPore test = new TestSuningPore(); 

        if (args.length == 0)
        {
            System.out.println("image file missing");
            return;
        }
        test.porno_recognize(test.readImage(args[0])); 
    }

    public TestSuningPore(){}

    private byte[] readImage(String path) throws Exception
    {
        File image = new File(path);
        FileInputStream fis = new FileInputStream(image);

        byte[] b = new byte[(int)image.length()];
        fis.read(b);
        fis.close();

        return b;
    }

    public String porno_recognize(byte[] imageByte)
    {
        if(null != imageByte)
        {
            SuningPore suning_pore = new SuningPore();
            boolean status = false;
            System.out.println("check status: " + status);

            if(!status)
            {    
                int load = suning_pore.load("/opt/pore/module/", "/opt/pore/lib/", "/opt/pore/log/");
                if(load != 1)
                {
                    System.out.println("Error: load failed!");
                    return "Error: load failed!";
                }
            }

            try
            {

                String words = suning_pore.start(imageByte);
                //String words = SuningPore.start("http://abc.efg.cn/hij.jpg", imageByte);  // this also works

                System.out.println("\nresulting words:\n" + words);

                return words;
            } catch (Exception e) {
                // TODO Auto-generated catch block
                e.printStackTrace();
            }
        }
        return null;
    }
}

测试jni接口

进入com上一级目录,并添加run.sh,添加如下代码:

cp ~/caffe/build/examples/porno_recognition/com_suning_pore_SuningPore.bin ./libSuningPore.so
cp ~/caffe/build/lib/libcaffe.so.1.0.0-rc3 .
javac -encoding UTF8 com/suning/pore/*.java
java -Djava.library.path=.  com.suning.pore.TestSuningPore $1

通过./run.sh xxx.jpg命令测试该接口

tips

依赖动态链接库问题

由于libSuningPore.solibcaffe.so.1.0.0-rc3依赖较多的动态链接库,可能需要指定动态链接库的路径,这时候把相关的动态链接库放到指定路径中,通过/etc/profile和source命令使该路径生效,并通过/etc/ld.so.conf文件设置include /opt/xxx/xxxldconfig来使系统更新相关路径的动态链接库。然后再重新编译caffe,通过ldd命令就可以查看这两个用到的动态库确实在新路径下找到了相关依赖。
需要注意的一点是,如果使用了mkl,注意可能需要一个libiomp5.so的文件,添加到相关动态链接库里就可以了。