表現学習時代の生成語彙論ことはじめ

1. 2014/10/16 PFIVZX^ ¿Ó½Ǝ»: ÐŇǖĦ!5;&D ĉPreferred Infrastructure Ù¦Ʋǁ (@unnonouno)

2. ƝǇŘ Ù¦Ʋǁ (@unnonouno) ! ŪåŇĭ¸GdVfuShXGÁǌ½Ǝ ! ĘƬĭ¸E/H05GKA' ! úß¸ýŵġIBMôǈŎPFI 8

3. mrep5c}P1KA%- ! řĹƑƚ8%3ÖĨåŇ4íƾ4A' ! pip install mrep 4Sf} 9

4. §:Ú ! word2vecPù8ƄŇ:¿Ó½ƎŹK:;7%P%A %- :

5. ¿Ó½Ǝ:ÚPň3Ĉ0-!5 ƣƵ.5ăJM3LƄ Ň:+&)PrWf} Į :8ƅ%ñDJM L:4;7 ;

6. ÐŇǖĦ Generative Lexicon (GL) <

7. ®:£Ć ! ÐŇǖĦ813:ƴƄ7ŉ ! ¿Ó½Ǝ5Êů:Ð5:¨Ē 18EA%3Œ:Ú OJ7, =

8. The Generative Lexicon [Pustejovsky95] ! James PustejovskyĀǆ ! ÄƸA4;ƟDA)Q4% - ! TimeML:Āǆ4EĐL ! Å; ¿Ó813:ÚP %Iƙ03-:4<0K %- > ÊÌ5¥% http://www.cs.brandeis.edu/~jamesp/

9. Ň:7J·¦é: [·¦05] ! ·¦ǍƆÐŇǖÊů Ħ ! ®ƟQ4LǕƥ ; _ ; ?

10. Æ8ħă8%-´ē ! ·¦ǍƆÐŇǖÊůĦ[·¦05]:Û12 ƽ ! James Pustejovsky, Introduction to Generative Lexicon. [Pustejovsky05] ! ěƲëGenerative Lexicon:ǋÇ [ě 06] 76

11. É:Êů813ă3BI ƛ:80- P8ƍ- ¥ƺ480- ! č8É5å03EĬî5%3:É5æŲ 5%3:ÉL 77

12. gR:Êů813ă3BI PsV4Ǒ0- J03«# ! wj5%3:gR5Ò'LÏ5%3:gR:Ê ůL ! é:É:ŗ5&I7!5ĸ!037

13. 78

14. ! ă3BL5ƄŇ;1:ƌ7LĎćP â03L!58Ñ2 ! ċOM8I03ÊůģO03L ! !:Êů:ģµ8;ęª:ĨƏÎ:I7E: ĐL ! %Eƌ7LåŇ4Eǃ-ÓłL 79

15. ÐŇǖĦ:ă ! 1:Ňǖ8ęª:ļŋPâ03L ! đǓ8Ī&3+:ļŋJƦ§ƌ7L ÊůÐ#M3L5ăL 7: É Ĭî:É æŲ:É $_

16. ŅÊŸƌĺŇ5êĺŇ;ĔĖ#ML ! êĺŇ;Êů¢8ŒJ:¨ĒPâ03LÔ çÇ ! -A-A&Ÿ870-Ň;ŸƌĺŇ5% 3ĔĖ#ML 7;

17. Ù¦;7*!ÚÝ8ŰůPâ03L Ī¹ƄŇGƣŇ~r}:ÊůPA Ʊ7ËÝŏê5ă3L %(¿þņ6&Êů %(¿þ&.6ņÊů 7<

19. Œģ+. 7>

20. ŗò:WU|Rļŋ 7? 1. ļÐ`Syǀ kg} 2. řĚŀKîÁ ǌ 3. ²¢ŀLĄĩ 'L 4. Æ±æB©3L ÿL @1AKN`KBC0 L4GM*!J&DIBQS[^T

21. Œ4!%-úßĠÇ7:

22. ŔŌ:ò;ş ŔŌ:ò;ngş ! ò;Ŋ¾'L5ƿǏ:Ƥø!!4; ²¢WU|RĐL ! +:-D;«:đ:Êů.5øǄ4 L 86

23. ƇőƿǏ:ƤøPįŁ'L5Yf87L

24. ƭǎä4ǅ3-Q.I Ɯö4 lhtS

25. 60/

26. ä4

27. Ɯö4

28. ƭǎ.Jä:8LƜö:.I ! Ɯö;ä8LE:5ƿǏ:ƤøPįŁ%3L ! ƶ8ärbg8703-Jƭǎ:^| o;J8ÀŪ 1Ƨ:åŇĭ¸5:PGK-9 87 ^[]U5@CLASSICAOP

29. 7Q4WU|RļŋPăL:

30. ! ê:ƄŇ:Êů8¨%3é8-I7ƿǏ: øūĢ'L ! +:-D-Eáơ#M-:I7¿Ó4Êů ª0-K+0K7đĖ:Êů87L!5ĐL ! !:ƿǏ:øJ³ï:đ4:ÊůÐ# ML:4;7 88

31. begin8;į×:ÊůÐ#ML begin the meeting begin a dance begin the book begin the movie ! +E+Ebegin:58;ƒGł4;

32. ! beginį×:ÊůPâ03L5ĈIKđǓ8Ī &3ÊůÐ#ML5ăL?Ū 89

33. ęÐco-composition begin

34. the book 1. begin:N8;łÒL;(.book;ł4 ;7 , 2. bookPł5%3øǄ%I`SqÍÈ 3. WU|RJúßPÂ03L ¶:Ðq^P 5Ž= 8:

35. cf. ƿǏ:ŝģŶ double x =

36. 10 1. double:N8;double:ŞÒL;(.10; double4;7 , 2. 10Pdouble:Ş5%3øǄ%I 3. Vxf:¯:úßPÂ03L ¶:Ðq^P13*",5Ž= 8;

37. ħă³;ÐŇǖĦ8;ĴK31:ļŋL ĝŃļŋ;ÐŇǖĦ:41ļŋ::11 ! Ƴļŋ ! łļŋ ! ĝŃļŋ ! Ňǖżƕļŋ ®;ėŷ%A'I 8<

38. !!A4:A5D ! ĥŇǖ8;+:ċOM8¨'LúßPâ0 3LWU|Rļŋ ! ŹŴ:ƄŇ5śćªK:øǄ478 !:WU|R:úßPċ03¥%ÊůPÐ 'LęÐ 8=

39. ŪåŇĭ¸:4ÐŇǖĦ;6MJċOM 3-

40. ! åŇĭ¸½:«430÷ ! ACL:«4300÷ 8> ÊÌ5Ĥ7

42. /H05!M;ģ ! 11:Ňǖ4!MJ:úß 5 . $./LHE, @A M5BAM

43. +3 8? 45. $.#/R%BFSY]W2M( $

44. WU|Rļŋ;ĨũdVfJ¾Ʈõ%- ! Web:íƾóČJ4ÿÐ%-m` PÜ¹%3ǐ'L¯ [Cimiano+07] ! ęĸǔ°5ðǒG¾ǒ:ƠÎPĝƯ8%3{ VX½ƎPƂ¹'L¯ [ĻŖ+12] 5;ċ:EƴƄ4;7#+ 96

45. Ĥ%ƖJ7%3BL ! ƄŇ6:I7ƃ:ƄŇ5Ƽ8ċOML 5!58'LüŭP±ųµ%3L ! 6:I7ƄŇ5ċOML8I03ƄŇ: ÊůEģOL 97

46. !:Ú6!4 §: Statistical Semantics:Ú8ǃ3L 98

47. !!4°ÐŇǖĦJŮM3 Êů:Ð:ÚP%A' 99

48. Statistical Semantics:#J Statistical Semantics is the study of "how the statistical patterns of human word usage can be used to figure out what people mean, at least to a level sufficient for information access”(ACL wiki) ! ŹŴƄŇ:Óǔ°76:Ĳè¢7ÎŃJ+:Ň: ÊůPăL ! ƚøGhz{}ibf76:¯¹JML 9:

49. ƄŇ:rWf}¿Ó ! §:Ú;ƄŚ:ƄŇ:ÊůP¿Ó%- ! ®§;ŹŴƄŇ5:¨Ē4ÊůģOLÚ7: 4/(J7Lo~]:Êů67 L:

50. 9;

51. ÐŇǖĦ5Statistical Semantics:¨Ē ! ÐŇǖĦ ! ĝ¡:đǓ8LƄŇ:Ó:ÊůPă3L ! 5+:I7ģµPĸ!'-D:Ňǖ:ļ ŋPă3L ! Statistical Semantics ! ŹŴƄŇ:Ɯ8I03Ɣ¢7+:ƄŇ:Êů Pă3L ! ƀ±¢7ƄŇ:Ó8'LÊů8åſ%37 ®J'LI 9<

52. Principle of compositionality (Frege1892] … the meaning of a complex expression is determined by the meanings of its constituent expressions and the rules used to combine them (wikipedia

54. 9=

55. ·#Q:{SgA5A03L https://speakerdeck.com/mamoruk/a-tensor-based-factorization-model-of-semantic-compositionality 9>

56. ¿ÓrWf}:Ð;ĵ%ĕIKƻĕķ [Mitchell+08] ! ĵ%ĕGƻĕ76rWf}:Ð¯Pıǉ ! ÇƑ"5:ƻĕMultiplicative¬0/F0- 9?

57. Statistical Semantics5:¨Ē ! Statistical Semantics§:£Ć;6/J 5å5#-(:rWf}PƮõ'LŎŠ ! Compositionality:ŎŠ;+:rWf}: :ŎŠ ĞP8øØEL%¿Ó½Ǝ:Ó 4ƁKKP)3L :6

58. Êů:ÐPƚ4¿Ó'L (MV-RNN) [Socher +12] ! ĥƄŇ8;rWf}5ƚĪ'L ! ŵƹ#ML5ƪ:ƚP¼:rWf}8ƻ3 #J8žĶř7ģŶ f PŦż¢8 ! RootA4ƗKľ'5đ:¿ÓrWf}õJML :7

59. GGd_}. [Cruys+13] ! ÆŇ¾ǒ²¢Ň:31Pd_}Pċ03æ) 3L:.5ăL :8

60. Dynamic Convolutional Neural Network [Kalchbrenner+14] ! CNNPċ03ƄŇ:A5AKPæB 3 ! ļđÞ4;7Żǃ:ŜČõJML

61. :9

62. ƇőCNN5compositionality ! Ŀ~Sy4;ŧD3ĳÏ¢ƄƷ7úß ! Ă:~Sy87L8¶M3+MJæBO#K IKƣƵ7Êů:LúßPûŭ%3L :: Layer1 Layer3 Layer5 [Zelier+14]OP'

63. åŇĭ¸;ĸÔņ 4 nW^}Tb[¤ǂ źřŸƑƄŇÚ ¿Ó

64. ƄŇo~]đ ø'LæB L M- :; M-

65. !:œŐ8ÐŇǖĦ:üPď)7 ! ÐŇǖĦ;CompositionalityP4űƘ% 3±ų2L-D8Ľ¸#M-¸Ħ5M7

66. ! šƕ½Ǝ76:m{aSvZofĸ+7 ®!+ĜŨ:üGŹŴ¦Pą'> :<

67. Ãà:Ú ƣƵ.5ăJM3LƄ Ň:+&)PrWf} Į :8ƅ%ñDJM L:4;7 :=

68. ęÐP¿ÓrWf}85K!C [Tsubaki+13] companyM JMrun run companyM)" run LOQ ! ƪƪ8ŕŢPſ@%3¥%rWf}PÿL ! 4-rWf}JÊů:ÐOML :> run companyM Mrun company LOQ runM JMcompany Mcompany '48

69. ƄŇ:Ð8'L1:ŁÔ ! ÐŇǖĦ ! åŇÓłPűƘ%3Awe}µ%I5'L īB ! Ť:×¸we} ! űƘ#M-åŇe`Pń'LI7we }:ŬƘ ! ¿Ó½Ǝųšƕ½Ǝų ! ļÐ5¿Ó:½ƎP85%3L !MJ:RSeR;ƞţ8¨Ē%3L :?

70. A5D ! ÐŇǖĦ5;

71. ! Ňǖ;8'LÀƊº7üŭJđǓ8Ī&3Ê ůÐ#ML5ăL ! Ň:Êů:Ð8¨'LŎŠ ! ƄŇ:¿ÓrWf}PÐ%3ƣ¿Ó:ÊůP ÿL ! ¿Ó½Ǝpv8ŀ03®İ03L åŇ½:ü5åŇĭ¸:¥%¯:ã E05ÕC

72. ;6

73. "ƓƢK5"$A%- ;7

74. ħăđƫ (1/3) ! [Pustejovsky95] James Pustejovsky. The Generative Lexicon. MIT Press. ! [·¦05] ·¦ǍƆ. (4+$. N%ư ! [Pustejovsky05] James Pustejovsky. Introduction to Generative Lexicon. Foundations of SemanticsƩĺ´ē. http://www.cs.brandeis.edu/~jamesp/classes/LING130/ ! [ě06] ěƲë. Generative Lexicon2. Ɛ:ğǊÍ:ŏÌƈ ;8

75. ħăđƫ(2/3) ! [Cimiano+07] Philipp Cimiano, Johanna Wenderoth. Automatic Acquisition of Ranked Qualia Structures from the Web. ACL2007 ! [ĻŖ+12] ĻŖƨ, ·ť, ěƲë. (4$ (!'&)

77. . 0. NLP2012. ! [Mitchell+08] Jeff Mitchell, Mirella Lapata. Vector-based Models of Semantic Composition. ACL2008. ! [Socher+12] Richard Socher, Brody Huval, Christopher D. Manning, Andrew Y. Ng. Semantic Compositionality through Recursive Matrix-Vector Spaces. EMNLP2012. ;9

78. ħăđƫ(3/3) ! [Cruys+13] Tim Van de Cruys, Thierry Poibeau, Anna Korhonen. A Tensor-based Factorization Model of Semantic Compositionality. NAACL2013. ! [Kalchbrenner+14] Nal Kalchbrenner, Edward Grefenstette, Phil Blunsom. A Convolutional Neural Network for Modelling Sentences. ACL2014. ! [Zelier+14] Matthew D. Zeiler, Rob Fergus. Visualizing and Understanding Convolutional Networks. ECCV2014. ! [Tsubaki+13] Masashi Tsubaki, Kevin Duh, Masashi Shimbo, Yuji Matsumoto. Modeling and Learning Semantic Co-Compositionality through Prototype Projections and Neural Networks. EMNLP2013 ;:

SlideShare for iOS

表現学習時代の生成語彙論ことはじめ

by Yuya Unno , Researcher at Preferred Infrastructure, inc.

on Oct 16, 2014

Statistics

Views

Actions

1 Embed 75

Accessibility

Categories

Upload Details

Usage Rights

Report content

表現学習時代の生成語彙論ことはじめ Presentation Transcript