Deep Learning in real world @Deep Learning Tokyo

Deep Learning in real world @Deep Learning Tokyo

1. 1. Deep  Learning  in  real  world: Automobile,  Robotics,  Bio  Science Daisuke  Okanohara hillbig@preferred.jp Preferred  Networks,  Inc. Preferred  Infrastructure,  Inc. March 20, 2016 @ DLT: Deep Learning Tokyo
2. 2. l  Preferred  Networks –  Founded  in  March  2014,  Offices:  Tokyo  and  San  mateo –  40  engineers  &  researchers –  Investors  :  NTT,  FANUC,  Toyota l  Transportation –  Autonomous  driving,  and  its  application –  Joint  work  with  Toyota,  Panasonic –  Manufacturing –  Intelligent  Robot,  Industrial  equipment –  Joint  work  with  FANUC –  Healthcare –  Genomic  analysis 2
3. 3. AutomotiveHumanoid Robot Preferred  Networksʼ’  positioning  in  AI:  Industrial  IoT 3 Consumer Industrial Cloud Device PhotoGame Text Speech Infrastructure Factory Robot Automotive Healthcare Smart City Industry4.0 Industrial IoT
4. 4. Parking  detection 4 l  Segmentation  +  Edge  detection
5. 5. Anomaly  Detection  from  Sensors 5 Deep  Learning  detects   abnormal  parts Detect   abnormal  signals Normal Abnormal Sensor  data  from   decelerators  in  robots
6. 6. 6 Deep  learning  based  method  can  detect   symptoms  of  failure  much  earlier Proposed  Method 経過時間 異異常スコア Detect  40  days   before  the  failure Threshold Previous  Method Elapsted  Time Detect  just  before   the  failure Robot failure Robot failure 15⽇日前
7. 7. Massive  prediction  of  (QSAR)   l  Drug  discovery –  Deep  learning  can  predict  cross-‐‑‒reactive,  side  effect,  and  toxicity  from  their structures  and  known  experimental  result. 7 Drugs Assays
8. 8. Deep  Learning  for  HealthCare Prediction Model Genome Drug Assay •  Multi-Modal Use different types of data Genome, Drug, Assay,
 Clinical dataset •  Multi-Task Learn similar different tasks at the same time to 
 enhance their capabilities Personalized
 Medicine Drug
 Discovery Diagnosis Clinical data
9. 9. chainer-‐‑‒DCGAN Anime  Image  Generation  from  scratch https://github.com/mattya/chainer-‐‑‒DCGAN 9
10. 10. 2  hours  later 10
11. 11. 1  day  later 11
12. 12. Future  direction
13. 13. 1.  We  need  “non-‐‑‒”supervised  learning l  Supervised  learning  is  very  successful –  However,  its  annotation  cost  is  large –  Even  human  cannot  annotate  for  difficult  problems –  E.g.  Segmentation  of  video,  massive  people  tracking,  carʼ’s  orientation l  We  have  unlimited  data  but  its  usage  is  still  limited –  Image,  video,  sound  and  other  sensor  (LIDAR)    +  Context  information l  Semi-‐‑‒supervised,  weakly  supervised,  one-‐‑‒shot  learning  is  promising –  Use  unlimited  unlabeled  data  and  very  few  labeled  data l  Reinforcement  learning  is  also  promising –  Self  training  (human  just  design  reward  rules) 13
14. 14. 2.  Machine  teaches  other  Machines l  Humans  learn  much  faster  than  machines –  “Better  than  a  thousand  days  of  diligent  study  is  one  day  with  a  great   teacher”  (Japanese  proverb) l  Trained  machine  can  teach  other  machines  in  several  ways –  Distillation  (Hinton+  2015) u  Imitating  teacherʼ’s  behavior  even  including  their  how  to  mistake u  E.g.  Convert  a  large  ensemble  model  into  a  small  model –  Privileged  Information  （Vapnik+  2014,  2015） u  Teacher  gives  hints  at  training  time,  and  student  use  them  to  learn  faster u  E.g.  Image  with  annotation  (this  is  the  image  of  20  male.) l  Mixing  different  knowledge  from  multiple  machines –  Gathering  compressed  knowledge  from  edge  devices  (car,  robotics) 14
15. 15. 3.  Other  objectives  for  learning l  We  need  another  training  objective  for  unsupervised  training –  Maximum  likelihood  estimation  is  not  good  strategy  for  high-‐‑‒dimensional  data u  The  estimation  of  gradients  of  the  partition  function  is  very  noisy –  GAN  (generative  adversarial  network)  works  quite  well  for  image  generation   task.  By  analyzing  GAN,  we  can  find  what  is  important  for  generative  model l  Humans  and  animals  seem  to  use  several  signals  for  learning  that   todayʼ’s  machines  cannot  use  yet. –  Predictability  seems  very  fundamental  for  unsupervised  learning  but  not  all –  Brain  seems  to  do  some  inference  (unconsciously)  so  that  it  explains  the   world,  that  makes  learning  signals  for  training  deep  layers    [Bengio+  2016] –  How  to  model  curiosity  (or  incentive  to  know  unknown  information)  in   machines  ?                                                                                              Thanks  ! 15