家庭健康服务机器人的关键技术-宋继强

7
视觉技术：从看见到理解
人脸，人 体识别
为了更好地与人交互
◦ Find people, recognize people, follow people ◦ Understand human activity ◦ Detect events ◦ Detect mood
物体识别
为了更好地与环境交互
障碍检测
Event buffer
Memory-based Personalization Framework
Event normalizatio n
(Padding, combine, augmentation)
Personalize d applications Pattern matchin g
User profiling
Memory engine
Raw Database
General mining
…
Event Base
Index Base
Pattern Base
Others
From various robot sensors, applications, connected devices
Viewer
9/25/2015
6
HSR实用化的关键技术
基本功能
（受控、半自主）
高级功能
（全自主）
• 听、说 • 看、识别 • 显示 • 被动运动 • 无线联网
能 用
• 行为理解 好 • 情绪感知 用 • 推理规划 • 主动运动 • 个性化服务
9/25/2015
INTEL CONFIDENTIAL ‐ INTERNAL USE ONLY
可以用，但并非完美：存在盲区、 缺乏颜色和全局信息、不便宜
9/25/2015
INTEL CONFIDENTIAL ‐ INTERNAL USE ONLY
9
深度摄像头的RGBD输入
Out of range
Reflective
Dark
静态场景
无纹理区域 低质量图像 光线变化
RGBD vSLAM的技术挑战
9/25/2015 INTEL CONFIDENTIAL ‐ INTERNAL USE ONLY
2
照料者危机(Caregiver Crisis)催生需求
• 照料者的困难 >50%有全职工作 需要住在20分钟响应 范围内
•
目前可用技术 智能手机应用 IOT设备 可穿戴设备
• 对老人、病人和照料者来说既是医疗问题，也是社会问题
Must-have IT Features
Sensing (7x24) Mobility (autonomous)
社交保持 (~15%)
•Search for latest news and their relatives/friends’ SNS information, filter and summarize, then present to elderly in an easy way •Help elderly post their status on SNS, record happy moment •Help elderly do video chat with relatives/friends
Home IOT Devices
AOAC module RGBD camera
WiFi + Big BT storage
Health Wearables
Microphone Other array sensors
跌倒检测?
Smart Phone Apps
9/25/2015
INTEL CONFIDENTIAL ‐ INTERNAL USE ONLY
Reminder Recommendation Anomaly detection
Social activity logging Location logging Conversation records logging Environmental data logging
Domainspecific Mining
Physical manipulation*
* Manipulation is nice to have in the first stage
9/25/2015
INTEL CONFIDENTIAL ‐ INTERNAL USE ONLY
4
智能手机应用、IOT和可穿戴设备的差距
智能手机应用
Sensing (7x24) Mobility (autonomous) Wireless/Internet access Cognitive computing Natural user interaction Telepresence (A/V/move) Physical manipulation
• •
Raw data: daily volume: ~730MB, annual volume: ~260GB Assuming robot refresh cycle = 3 years, local storage size should > 1TB
9/25/2015
INTEL CONFIDENTIAL ‐ INTERNAL USE ONLY
动态场景
小变化
沙发上有/没有衣服 移动了桌子上的杯子
远距离或深色区域 (no depth) 视野突变 对策:
图像增强 与其他输入结合 改进算法
大变化
门开着/关着 窗帘开着/关着 家居移位、增减
对策:
更多训练数据 使用物体级别的高层特征 与其他输入结合
Wireless/Internet access Cognitive computing Natural user interaction Telepresence (A/V/move)
紧急救助 (<10%)
•Detect anomaly from visual, aural and other input (like fall down, painful facial expression, big sound, etc.) •Do something according to first-aid guide for robots (like call someone, open window if poison gas is around) •Allow remote telepresence to let human judge and operate
LIDAR与vSLAM对比
Capability Location error (cm) Tolerate lightness change Can handle black area Can handle glass area Can handle texture-less area Can handle thin object Color information Map information Real-time Cost Easy to deploy LIDAR <1 or ∝ Yes No No Yes No No Yes Yes Expensive Yes Takeaway: Neither is perfect, but combining them can get all “YES”. vSLAM <5 or ∝ Hard Yes Yes No Yes Yes Yes No Median to low Yes
Think with memory
Jeff Hawkins: “memory + prediction” generates real intelligence. ‐‐ 《On Intelligence》 ‐‐《人工智能的未来》
13
家庭健康服务机器人的多维记忆
Environment Activity Video, image
◦ Object identification and manipulation ◦ Interplay with SLAM to help navigation
9/25/2015
INTEL CONFIDENTIAL ‐ INTERNAL USE ONLY
8
移动技术：在动态环境中自主移动
LIDAR‐based SLAM • 目前最便宜的RPLidar: RMB2400 • 360°, 6M, one‐plane map
Conversation Health
9/25/2015
INTEL CONFIDENTIAL ‐ INTERNAL USE ONLY
14
记忆的原始数据量估计
Data source Fixed-interval picture (day) Fixed-interval picture (night) Event-triggered picture Activity video (avg. 30 seconds) Conversation (avg. 60 seconds) Health data Environmental data Unit size (MB) 0.5 0.5 0.5 2 0.5 0.001 0.001 Frequency/hour 60 15 ~10 2 6 0.2 60 Hours 16 8 16 16 16 15 24 Daily volume (MB) 480 60 80 64 48 0.3 1.44
健康可穿戴设备
Sensing (7x24) Mobility (autonomous) Wireless/Internet access Cognitive computing Natural user interaction Telepresence (A/V/move) Physical manipulation
家庭IOT设备
Sensing (7x24) Mobility (autonomous) Wireless/Internet access Cognitive computing Natural user interaction Telepresence (A/V/move) Physical manipulation
15
基于记忆的个性化服务
Data logging
Health data collection Daily routine logging
(eating,wk.baidu.comdrinking…)
Event analyzer Event analyzer Event analyzer Event analyzer Event analyzer Event analyzer
注：3颗星表示功能足够好
局限： 没有一个是多传感、自主移动、本地智能运算 和远程呈现的自然整合。
9/25/2015
INTEL CONFIDENTIAL ‐ INTERNAL USE ONLY
5
HSR是终极解决方案
HSR
Aging care usages
Speech UI Core ix CPU Motion platform Visual UI Social Personalization I/O HW ACC Plan & control
INTEL CONFIDENTIAL ‐ INTERNAL USE ONLY
家庭健康服务机器人的关键技术
宋继强 英特尔中国研究院
养老服务成为全球问题
年龄> 65 现在：2.12亿 年龄 > 60 连续增长·15 年 到底峰值4亿
2010年全国人口统计
• 巨大社会问题 巨大市场机会（政府支持） • 全球万亿级市场，但先进IT技术渗透率很低 • 绝大部分老人及其家人偏向于居家养老
9/25/2015 INTEL CONFIDENTIAL ‐ INTERNAL USE ONLY
3
家庭健康服务机器人（HSR）的功能需求
健康与情感关怀 (~80%)
•Voice chat with elderly, with memory-based personalization •Use words, graphics, sounds, lights to improve elderly’s mood •Recommend proper physical activities to elderly •Remind their relatives and friends to give care