Your SlideShare is downloading.
×
×
![]()
Saving this for later?
Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply
Transcript
- 1. 1 Preferred Infrastructure 2015/06/19
- 2. ! -2008 ! ! 2008-2011 t 1 ! t ! 2011- 1 t ! t ! ! Jubatus NLP 12014- t 12015,
- 3. NLP 1YANS ! YANS 19 ! ! -‐‑‒19/3 9/5 ! YANS 13 ! ! "
- 4. 1 ! !
- 5. 2 ! ! ! ! ! !
- 6. ! ! ! ! t PFI
- 7. 3 1. 2. t 3.
- 8. 1.
- 9. ! ! ---- ---- ! Web EC t Web ! 1CGM ---- ---- ! SNS1Twitter Facebook ! 1Line ! t
- 10. ! ! ! ! ! ! t WWW ! Blog ! SNS
- 11. ! EC t ! ! ! ! ! ! SNS ! ! t
- 12. ! EC t ! 11000 ! ! ! ! ! SNS ! !
- 13. t ! ! ! ! ! ! SNS ! !
- 14. vs vs vs ! ! etc. ! UI
- 15. ! ! ! ! ! ! 1 1 SNS ! t
- 16. ! ! ! ! ! ! 1
- 17. 2. t
- 18. 24 http://www.soumu.go.jp/johotsusintokei/whitepaper/ja/h24/html/nc122310.html
- 19. ! !
- 20. t t : Siri https://www.apple.com/jp/ios/siri/ http://v-assist.yahoo.co.jp/ https://www.nttdocomo.co.jp/service/information/shabette_concier/
- 21. Gunosy, SmartNews http://gunosy.com https://www.smartnews.be/
- 22. 11/2 ! ! 1970 t PC ! ! ! 2000 ! t
- 23. 12/2 ! ! RSS 2000 ! ! 1 ! 2000 !
- 24. ! ! t ! ! ! !
- 25. t ! ! t ! ! t ! ! ! ! ! !
- 26. t
- 27. t ! !
- 28. ! ! <12:00 :25
- 29. t ! ! ! ! ! ! t
- 30. t ! t ! ! t ! t ! ! t
- 31. 3.
- 32. ! ! 1993: [Brown+93] ! 1996: [Berger+96] ! 2001: [Lafferty+01] ! t ! 2003: Latent Dirichlet Allocation [Blei+03] ! 2006: Pitman-Yor language model [Teh06] ! ! 2006: [Clarke+06][Riedel+06] ! 2010: [Koo+10][Rush+10] ! ! 2003: Neural language model [Bengio+03] ! 2010: Recurrent Neural Network [Mikolov+10] ! 2013: Skipgram Model (word2vec) [Mikolov+13]
- 33. ! ! 90 2010 !
- 34. ! !
- 35. 1. ! 2011: 30% # 20% ! 2012: 26% # 16% http://image-net.org/challenges/LSVRC/2012/ilsvrc2012.pdf
- 36. 2. ! 2012/3: Google Hinton DNNresearch ! 2012/4: Baidu Institute of Deep Learning ! 2012/8, 10: Yahoo! IQ Engines LookFlow ! 2012/12: Facebook AI Lab LeCun ! 2014/1: Google DeepMind ! 2014/5: Andrew Ng Baidu ! 2014/8: IBM SyNAPSE
- 37. 3. ! 2014/1 ! 2014/10 -‐‑‒ ! 2015/06
- 38. 4. t ! ! Sequence-to-sequence (2014, Google) ! ! Deep Q-Networks (2013, DeepMind) ! ! Variational Auto-Encoder (2014, UvA) ! Stochastic Backprop (2014, DeepMind) ! ! Neural Turing Machines (2014, DeepMind) ! ! Memory Networks (2014, Facebook) ! ! Show and Tell (2015, Google)
- 39. ! ! ! ! ! ! !
- 40. ! t ! ! ! 1 ! !
- 41. Neural Network Language Model (NNLM) [Bengio +03] ! N NN ! N-1
- 42. Recurrent Neural Network Language Model (RNNLM) [Mikolov+10] ! t-1 t ! NNLM N ! ! http://rnnlm.org
- 43. Skip-gram (word2vec) [Mikolov+13] ! ! 1CBOW ! Analogical reasoning ! 1word2vec
- 44. (MV-RNN) [Socher +12] ! ! f ! Root
- 45. Sequence-to-sequence learning [Sutskever+14] ! !
- 46. Show and Tell [Vinyals+15] ! ! CNN RNN CNN CNN RNN
- 47. Memory networks [Weston+15][Sukhbaatar+15] ! !
- 48. ! ! ! 1 ! End-to-end ! ! !
- 49. ! libsvm, liblinear ! ! JUMAN, Chasen, MeCab ! ! Moses (GIZA++) ! ! Stanford CoreNLP ! ! word2vec ! Skipgram ! Theano, Caffe, Torch ! t !
- 50. Chainer v1.0.0 2015/06/09 Powerful Flexible Intuitive
- 51. Chainer ! Chainer ! Python h(f(x) + g(x)) x f g h+
- 52. ! ! 1 ! ! ! ! ! Chainer
- 53. t
- 54. PFI 2010 Xappy12012 ! 2012 Gunosy ! ! ! t
- 55. Xappy t ! ! Blog ! Twitter Facebook SNS ! ! 4G ! t ! ! etc. ! GUI t
- 56. ! ! ! ! t ! PC t ! ! ! ! 10 !
- 57. (1/5) ! [Brown+93] Peter F. Brown, Vincent J. Della Pietra, Stephen A. Della Pietra, Robert L. Mercer. The mathematics of statistical machine translation: parameter estimation. Computational Linguistics Vol. 19 (2), 1993. ! [Berger+96] Adam L. Berger, Vincent J. Della Pietra, Stephen A. Della Pietra. A Maximum Entropy Approach to Natural Language Processing. Computational Linguistics, Vol. 22 (1), 1996. ! [Lafferty+01] John Lafferty, Andrew McCallum, Fernando C. N. Pereira. Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data. ICML2001.
- 58. (2/5) ! [Blei+03] David M. Blei, Andrew Y. Ng, Michael I. Jordan. Latent Dirichlet Allocation. JMLR Vol. 3, 2003. ! [Teh06] Yee Whye Teh. A Hierarchical Bayesian Language Model based on Pitman-Yor Processes. ACL 2006. ! [Clarke+06] James Clarke, Mirella Lapata. Constraint-Based Sentence Compression: An Integer Programming Approach. COLING/ACL 2006. ! [Riedel+06] Sebastian Riedel, James Clarke. Incremental Integer Linear Programming for Non-projective Dependency Parsing. COLING/ACL 2006.
- 59. (3/5) ! [Koo+10] Terry Koo, Alexander M. Rush, Michael Collins, Tommi Jaakkola, David Sontag. Dual Decomposition for Parsing with Non-Projective Head Automata. EMNLP 2010. ! [Rush+10] Alexander M. Rush, David Sontag, Michael Collins, Tommi Jaakkola. On Dual Decomposition and Linear Programming Relaxations for Natural Language Processing. EMNLP 2010. ! [Bengio+03] Yoshua Bengio, Réjean Ducharme, Pascal Vincent, Christian Jauvin. A Neural Probabilistic Language Model. JMLR, 2003.
- 60. (4/5) ! [Mikolov+10] Tomas Mikolov, Martin Karafiat, Lukas Burget, Jan "Honza" Cernocky, Sanjeev Khudanpur. Recurrent neural network based language model. Interspeech, 2010. ! [Mikolov+13] Tomas Mikolov, Kai Chen, Greg Corrado, Jeffrey Dean. Efficient Estimation of Word Representations in Vector Space. CoRR, 2013. ! [Socher+12] Richard Socher, Brody Huval, Christopher D. Manning, Andrew Y. Ng. Semantic Compositionality through Recursive Matrix-Vector Spaces. EMNLP2012. ! [Sutskever+14] I. Sutskever, O. Vinyals, Q. V. Le. Sequence to Sequence Learning with Neural Networks. NIPS 2014.
- 61. (5/5) ! [Vinyals+15] O. Vinyals, A. Toshev, S. Bengio, D. Erhan. Show and Tell: A Neural Image Caption Generator. arXiv:1411.4555, 2014. ! [Weston+15] J. Weston, S. Chopra, A. Bordes. Memory Networks. ICLR 2015 ! [Sukhbaatar+15] S. Sukhbaatar, A. Szlam, J. Weston, R. Fergus. End-To-End Memory Networks. arXiv:1503.08895, 2015.
Be the first to comment