智能信息物理系统时代的机器人技术创新—张建伟

Cybernetics: Systems, 2016 46(7): 969-979. Chang Liu, Fuchun Sun, Changhu Wang, Feng Wang, Alan Yuille, “MAT: A Multimodal Attentive Translator for Image Captioning”, IJCAI 2017 (oral)
© it s OWL Clustermanagement GmbH | 3 | 06.12.2012 10
www.its-owl.de
11
t
o R o C en ,o t v C en o o x r en
T
T o s b g x i v -, p a R -, xo
11
From industrial robot to cobot
like a deep neural network
Outputs
1 (forward) 2 (backward) 3 (rotate right) 4 (rotate left) 5 (extend finger)
Sensory Apparatus
Motor Apparatus
Previous Forecasts
+ Semantic description + Networking, Internet (M2M) + Wireless communikation
Internet of data and services
+
Internet of things
+ IP-capabilities
+ Sensors, actuators + Integration enabled highperformance microcomputers
37
C. Crossmodal learning in human-machine interaction
C4
Dr. Zhiyuan Liu
Dr. Stefan Heinrich
✓
1. 2. 3. Heinrich, S., Weber, C., and Wermter, S., Xie, R., Lin, Y., Liu, Z. (2016). Crossmodal language grounding, learning, and teaching. In CoCo@NIPS2016. Xie, R., Liu Z., Luan, H., Sun, M. (2017). Image-embodied Knowledge Representation Learning. In IJCAI 2017. Niu, Y., Xie, R., Liu, Z., Sun, M. (2017). Improved Word Representation Learning with Sememes. In ACL 2017.
Instruc7ons+ Environment+
Planning+
Ac7vi7es+
51
Architecture and Software Design for a Service Robot in an Elderly-Care Scenario. Hendrich, Bistry, Zhang. Engineering 2015, Vol. 1 Issue (1) : 27 -35 DOI: 10.15302/J-ENG-205007 52
43
+
-
-
+ +
44
45
à
Ula
46
47
48
IEEE ROBIO 2013 Best Paper Award
49
/
50
Background+ Knowledge+ Learning+ Rule+Extrac7on+ Conceptualisa7ons+ + Experiences+ Recording+
Source:Festo, Kuka
12
13
14
Technical Commitees IEEE Robotics and Auomation Society
Aerial Robotics and Unmanned Aerial Vehicles Agricultural Robotics and Automation Algorithms for Planning and Control of Robot Motion Automation in Health Care Management Automation in Logistics Autonomous Ground Vehicles and Intelligent Transportation Systems Bio Robotics Cognitive RoObotics Computer & Robot Vision Cyborg & Bionic Systems Energy, Environment, and Safety Issues in Robotics and Automation Haptics Human Movement Understanding Human-Robot Interaction & Coordination Humanoid Robotics Marine Robotics Mechanisms and Design Micro/Nano Robotics and Automation Mobile Manipulation Model-Based Optimization for Robotics Multi-Robot Systems 15
36
B. Efficient crossmodal generalization and prediction
B5 Prof. Jianwei Zhang
Prof. Fuchun Sun
Sun Fuchin
Liu Chunfang
Huang Wenbing, Zhang Jianwei, “Object classification and grasp planning using visual and tactile sensing,” IEEE Trans. Systems, Man, and
6 01
8 .
1
2
• • • •
-
2
• • • • •
3
2.0
4
5
3D
6
IMA(Iontronic microdroplet array )
R. Li, et.al , “Supercapacitive Iontronic Nanofabric Sensing,” Adv Mater, vol. 29, 1700253, pp. 1-8
O - B
I
… …
34
16
7 . 3 21 2 7 M 6 46
5
5
35
A. Dynamics of crossmodal adaptation
A2
,
Prof. Jisong Guan
Prof. Claus Hilgetag
✓
1.
,
Ding X, Liu S, Tian M, Zhang W, Zhu T, Li D, Wu J, Deng H, Jia Y, Xie W, Xie H, Guan JS(2017). Activity-induced histone modifications govern Neurexin-1 mRNA splicing and memory preservation. Nat Neurosci. 2017 May; 20(5):690-699. doi: 10.1038/nn.4536.
1 Computer Many Users
1 Computer 1 User
Z e it
1941 Z3
1960
1980
2000
2020
17
Hale Waihona Puke Baidu
-
-
18
19
The Progression of Automation Industrial Automation (Machines) Information Automation (Software) Supervised Learning Reinforcement Learning Continual Learning
Forecasts
24
25
26
-
27
R-CNN
ORB
SLAM
Semantic SLAM
28
Ø
0 0
2
Ø
0 0
Ø
2 . 0
0
0
Ø
0
29
-
30
-
31
-
32
B 9
R
• •
L L
F C
6 .-2
9 T S
-6 1 0
www.crossmodal-learning.org
33
1
1 5 : 2 : 4 32 . 6A 32 5C
Zhang et al., 2015
Shi et al., 2014
Cheng et al., 2015
Zhang et al., in preparation
Liang et al., 2015a; 2015b
In progress
22
23
2 3
Inputs Black Box Supervised learner
38
(1)
(2)
(3)
(4)
CPS
39
40
Modular robots
41
42
Energy-based Domain Adaptation
Motivation 1. Feature from the Pretrained Deep Models (FPDM) has good transfer characteristic.2. 2. Observing from the view of energy, it will be seen that the projection of the given target sample emerges some kind of energy distribution characteristics
16
-
Intelligent Environments
Embedded Computers
Smart City
Smart Factory
Main Frame
PC
Smart Phone
90% of all Processors are hidden 1 User Many Computers 8,5% Growth 17 Billion Turn Over
Technical Commitees IEEE Robotics and Auomation Society
Neuro-Robotics Systems Performance Evaluation & Benchmarking of Robotic and Automation Systems Rehabilitation and Assistive Robotics RoboCup Robot Ethics Robot Learning Robotic Hands, Grasping and Manipulation Robotics and Automation in Nuclear Facilities Safety, Security and Rescue Robotics Semiconductor Manufacturing Automation Smart Buildings Soft Robotics Software Engineering for Robotics and Automation Space Robotics Surgical Robotics Sustainable Production Automation Telerobotics Wearable Robotics W hole-Body Control
20
Machine Learning Paradigms
21
Zhang et al., 2016
Hu et al., 2012, 2014
BI
AI
Zhang et al., 2003 Hu, zhang et al., 2014
Liang et al., 2013; Wu et al., 2013; Li et al., 2015
7
8
(
)
(
9
Many users, 1 computer
Mainframe
1 user, 1 computer
Data Warehouses, Internet, PC
1 users, many computers
Big Data, Cloud Computing, Smart Devices Cyber-Physical Systems (CPS) Imbedded Systems Physical Objects, Equipment, …