技术控

    今日:110| 主题:49431
收藏本版 (1)
最新软件应用技术尽在掌握

[其他] Test NLP-as-a-service providers with Bunt

[复制链接]
远昼 发表于 2016-10-1 23:33:32
78 0

立即注册CoLaBug.com会员,免费获得投稿人的专业资料,享用更多功能,玩转个人品牌!

您需要 登录 才可以下载或查看,没有帐号?立即注册

x
BUNT (Bot UNderstanding Testbed)

  What is BUNT ?

  BUNT is a testbed for NLP engines, typically used in chatbots.
  Find out more about the story behind BUNT    on this blog post  
  What BUNT can do for you ?

  BUNT can evaluate performance on:
  
       
  •       languages: French and English are supported. Other languages can be added through modules.   
  •       corpus: Different default corpus of testing data are provided (small talk, misspellings, …) - other corpus can be added by the user   
  •       apis: 3 NLP-as-a-service providers are supported: api.ai, luis.ai, and recast.ai. Other providers can be added through plugins  
  How does BUNT work ?

  
       
  • Put your credentials in      credentials.py:  
  1. LUIS_KEY = 'my_luis_key'
复制代码

       
  • Select your settings in      settings.py:  
  1. ACTION = 'comparator'
  2. METRICS = ['accuracy', 'error_3_penalized']
  3. CRITERIA = {
  4.     'en': ['mails_en', 'misspellings_en'],
  5.     'fr': ['mails_fr', 'misspellings_fr']
  6. }
复制代码

       
  • Run      run.py:  
  1. python run.py
复制代码
How can I add my own NLP service ?

  
       
  • Create a class Manager inheriting ApiManager from      api.py. (See      api.pyfor more details):  
  1. class MyAPIManager(ApiManager):
  2.     def __init__(self):
  3.         pass
  4.     def fit(self, df_train):
  5.         pass
  6.     def predict(self, x_list):
  7.         pass
复制代码

       
  •       Add the name of your API in the settings file (in API_HANDLED)
       
  •       In        api_builder.py, add a case with your api (in build_api function)      
      
  How can I add my own metric ?

  
       
  • Add your metric in score_metric method of      scorer.py:  
  1. if metric == 'your_metric_name':
  2.     # this example is the accuracy metric
  3.     n_all = n_found + n_fallback + n_error
  4.     return n_found / float(n_all)
复制代码

       
  • Add your metric name in      settings.pyin METRICS_HANDLED:  
  1. METRICS_HANDLED = ['accuracy',..., 'your_metric_name']
复制代码
How can I add my own test set ?

  
       
  •       Create a csv file (tab-separated) containing 2 rows
          The utterances are in the row called        sentenceand the intents are in the other row called        intent(you can see one of the files)      
       
  •       Add your file in        criteria/languagefolder.      
       
  •       Add the name of your file (without the        .csv) in CRITERIA in        settings.pywhen you want to use it:      
      
  1. CRITERIA = {
  2.     'en': ['smalltalk_en', ... 'your_english_file'],
  3.     'fr': ['smalltalk_fr', 'mails_fr', 'your_french_file']
  4. }
复制代码
友荐云推荐




上一篇:Swift 模式匹配学习笔记1
下一篇:用大白话谈谈XSS与CSRF
酷辣虫提示酷辣虫禁止发表任何与中华人民共和国法律有抵触的内容!所有内容由用户发布,并不代表酷辣虫的观点,酷辣虫无法对用户发布内容真实性提供任何的保证,请自行验证并承担风险与后果。如您有版权、违规等问题,请通过"联系我们"或"违规举报"告知我们处理。

*滑动验证:
您需要登录后才可以回帖 登录 | 立即注册

本版积分规则

我要投稿

推荐阅读

扫码访问 @iTTTTT瑞翔 的微博
回页顶回复上一篇下一篇回列表手机版
手机版/CoLaBug.com ( 粤ICP备05003221号 | 文网文[2010]257号 )|网站地图 酷辣虫

© 2001-2016 Comsenz Inc. Design: Dean. DiscuzFans.

返回顶部 返回列表