Konrad 'ktoso' Malawski
GeeCON 2014 @ Kraków, PL
Konrad `@ktosopl` Malawski @ Reactive Systems Meetup
streams
How reactive...
Konrad `ktoso` Malawski
(we’re renaming soon!)
Akka Team,
Reactive Streams TCK,
Maintaining Akka Http
TypesafeのKonrad `kto...
Konrad `@ktosopl` Malawski
akka.io
typesafe.com
geecon.org
Java.pl / KrakowScala.pl
sckrk.com / meetup.com/Paper-Cup @ Lon...
Nice to meet you!
Who are you guys?
はじめまして!
Agenda for today:
• Story & landscape
• The Reactive Streams Protocol
• Akka Streams / Demo
• Akka Http / Demo
• Q/A?
本日のア...
Reactive Streams - story: early FRP
http://blogs.msdn.com/b/rxteam/archive/2009/11/17/announcing-reactive-extensions-rx-fo...
Reactive Streams - story: 2013’s impls
~2013:
Reactive Programming
becoming widely adopted on JVM.
- Play introduced “Iter...
Reactive Streams - story: 2013’s impls
Play Iteratees – pull back-pressure, difficult API
http://blogs.msdn.com/b/rxteam/ar...
Reactive Streams - Play’s Iteratees
def fold[B](
done: (A, Input[E]) => Promise[B],
cont: (Input[E] => Iteratee[E, A]) => ...
Reactive Streams - expert group founded
October 2013
Roland Kuhn (Akka) and Erik Meijer (Rx .NET) meet in Lausanne,
while ...
Reactive Streams - expert group founded
October 2013
Roland Kuhn (Akka) and Erik Meijer (Rx .NET) meet in Lausanne,
while ...
Reactive Streams - expert group founded
October 2013
Roland Kuhn (Akka) and Erik Meijer (Rx .NET) meet in Lausanne,
while ...
Reactive Streams - expert group founded
October 2013
Roland Kuhn (Akka) and Erik Meijer (Rx .NET) meet in Lausanne,
while ...
October 2013
Roland Kuhn (Akka) and Erik Meijer (Rx .NET) meet in Lausanne,
while recording “Principles of Reactive Progra...
Reactive Streams - story: 2013’s impls
2014–2015:
Reactive Streams Spec & TCK
development, and implementations.
1.0 releas...
2014–2015:
Reactive Streams Spec & TCK
development, and implementations.
1.0 released on April 28th 2015,
with 5+ accompan...
in a few words:
• Toolkit for building scalable distributed / concurrent apps.
• High Performance Actor Model implementati...
Why back-pressure?
?
なぜ背圧制御なのでしょうか?
Why back-pressure?
So you’ve built your app and it’s awesome.
皆さんはアプリを構築してきました、
それは素晴らしいことです。
Why back-pressure?
Let’s not smash it horribly under load.
負荷をかけすぎて、壊さないようにしましょう。
What is back-pressure?
?
背圧制御(back pressure) とは何でしょうか?
What is back-pressure?
背中(back)が痛くて死にそうだ!
No no no…!
Not THAT Back-pressure!
No no no…!
Not THAT Back-pressure!
What is back-pressure?
いやいやいや...! その back pressure で...
Publisher[T] Subscriber[T]
Back-pressure explained
背圧制御の説明
Fast Publisher Slow Subscriber
What if…?
もしも、1秒間に100の命令を処理する高速なPublisherと、
1秒間に1つの命令しか処理しない低速なSuscriberだったら?
Push + NACK model
Push と NACK モデル
Push + NACK model
Subscriber usually has some kind of buffer.
Subscriber は普通ある種のバッファを持っています。
Push + NACK model
Push + NACK model
What if the buffer overflows?
Push + NACK model
もしバッファがオーバーフローしたら何が起こるでしょう?
Use bounded buffer,
drop messages + require re-sending
Push + NACK model
大きさが限られたバッファを使い、
メッセージを廃棄し、再送を要求します。
Kernel does this!
Routers do this!
(TCP)
Use bounded buffer,
drop messages + require re-sending
Push + NACK model
カーネルがこれを...
Increase buffer size…
Well, while you have memory available!
Push + NACK model
バッファサイズを増やしてみましょう...
そう、使用できるメモリがある限りは!
Push + NACK model
メモリ不足
Negative ACKnowledgement
NACKing
Buffer overflow is imminent!
バッファオーバーフローが今にも起こりそうです!
Telling the Publisher to slow down / stop sending…
NACKing
Publisher にスローダウンするか、送信を止めるよう知らせます...
NACK did not make it in time,
because M was in-flight!
NACKing
NACK は間に合いませんでした、
メッセージが飛んできている最中だったからです!
What if…
We don’t need to back-pressure, because:
speed(publisher) < speed(subscriber)
We need low-overhead for “happy cas...
No problem!
Fast Subscriber => no problem
高速な Subscriber であれば、問題ありません!
Back-pressure?
Reactive-Streams
=
“Dynamic Push/Pull”
Fast Subscriber => no problem
背圧制御? Reactive Streams = "動的な Push/Pul...
Just push – not safe when Slow Subscriber
Just pull – too slow when Fast Subscriber
Reactive Streams: “dynamic push/pull”
...
Solution:
Dynamic adjustment
Just push – not safe when Slow Subscriber
Just pull – too slow when Fast Subscriber
Reactive ...
Slow Subscriber sees it’s buffer can take 3 elements.
Publisher will never blow up its buffer.
Reactive Streams: “dynamic ...
Fast Publisher will send at-most 3 elements.
This is pull-based-backpressure.
Reactive Streams: “dynamic push/pull”
高速なPub...
Fast Subscriber can issue more Request(n),
before more data arrives!
Reactive Streams: “dynamic push/pull”
高速なSubscriberは、...
Fast Subscriber can issue more Request(n),
before more data arrives.
Publisher can accumulate demand.
Reactive Streams: “d...
Publisher accumulates total demand per subscriber.
Reactive Streams: accumulate demand
Publisherはsubscriberごとに需要の合計を蓄積します。
Total demand of elements is safe to publish.
Subscriber’s buffer will not overflow.
Reactive Streams: accumulate demand
需要の...
http://reactive-streams.org
We want to make different implementations
co-operate with each other.
Reactive Streams: Inter ...
http://reactive-streams.org
We want to make different implementations
co-operate with each other.
Reactive Streams: Inter ...
RS is NOT a “daily use”, “end-user” API.
It’s an SPI - Service Provider Interface.
Reactive Streams: Inter-Op
https://en.w...
EmbeddedApp.fromHandler(new Handler {
override def handle(ctx: Context): Unit = {
// RxJava Observable
val intObs = Observ...
EmbeddedApp.fromHandler(new Handler {
override def handle(ctx: Context): Unit = {
// RxJava Observable
val intObs = Observ...
Akka Streams
streams
Akka Streams & HTTP
streams
& HTTP
Akka Streams in 20 seconds:
// types:
Source[Out, Mat]
Flow[In, Out, Mat]
Sink[In, Mat]
// generally speaking, it's always...
Akka Streams in 20 seconds:
// types: _
Source[Int, akka.NotUsed]
Flow[Int, String, akka.NotUsed]
Sink[String, Future[Stri...
Akka Streams in 20 seconds:
// types: _
Source[Int, Unit]
Flow[Int, String, Unit]
Sink[String, Future[String]]
Source.sing...
Akka HTTP
Joint effort of Spray and Akka teams.
Complete HTTP Server/Client implementation.
Soon prod ready, developed ~1....
It’s turtles buffers all the way down!
親亀の上に子亀、子亀の上に孫亀がのるかのように、バッファの
次にバッファ、その次にまたバッファ... と無数につらなります!
Streaming from Akka HTTP
Akka HTTP からのストリーミング
Streaming from Akka HTTP
Akka HTTP からのストリーミング
Streaming from Akka HTTP
Streaming from Akka HTTP
Streaming from Akka HTTP
No demand from TCP
=
No demand upstream
=
Source won’t generate tweets
TCPからの要求がない = 上流の要求がない
= S...
Streaming from Akka HTTP
No demand from TCP
=
No demand upstream
=
Source won’t generate tweets
=>
TCPからの要求がない = 上流の要求がない
...
Streaming from Akka HTTP
No demand from TCP
=
No demand upstream
=
Source won’t generate tweets
=>
Bounded memory
stream p...
Client / Server “JSON Streaming” demo
Demo time
クライアント/サーバの "JSONストリーミング" デモ
Akka Streams
Hidden powers:
Parallelism
&&
Pipelining
Akka Streams の隠されたパワー:
並列処理(Parallelism)とパイプライン
Pipelining Pancakes
http://doc.akka.io/docs/akka-stream-and-http-experimental/1.0/scala/stream-parallelism.html
Pipelining
Pipelining
Pipelining
// Takes a scoop of batter and creates a pancake with one side cooked
val fryingPan1: Flow[ScoopOfBatter, HalfC...
Parallelism
???
Parallelism
Parallelism
val fryingPan: Flow[ScoopOfBatter, Pancake, Unit] =
Flow[ScoopOfBatter].map { batter => Pancake() }
val pancak...
Parallelism
val fryingPan: Flow[ScoopOfBatter, Pancake, Unit] =
Flow[ScoopOfBatter].map { batter => Pancake() }
val pancak...
Parallelism
val fryingPanFun: ScoopOfBatter ⇒ Future[Pancake] =
batter ⇒ Future.successful(Pancake())
val pancakeChef: Flo...
Pipelining && Parallelism
Parallelism
&&
Pipelining
do the heavy-work for you.
並列処理とパイプラインは、あなたのために
きつい作業をやってくれます。
10/26/2015 spray-can: add websockets support (client & server) · Issue #134 · spray/spray
Pull requests Issues GistThis re...
Spray’s most requested feature ever:
WebSockets
path("ws") {
val handler: Flow[Message, Message] = ???
handleWebsocketMess...
Spray’s most requested feature ever:
WebSockets
path("ws") {
val handler: Flow[Message, Message] = ???
handleWebsocketMess...
Spray’s most requested feature ever:
WebSockets
path("ws") {
val handler = Flow.fromSinkAndSource(
Sink.ignore,
Source.sin...
Summing up…
Summing up…
要約...
buffers, buffers everywhere!
https://dev.twitter.com/streaming/overview/request-parameters#stallwarnings
あらゆるところにバッファが!
JEP-266 – soon…!
public final class Flow {
private Flow() {} // uninstantiable
@FunctionalInterface
public static interfac...
Back-pressure as a feature
特徴(目玉機能)としての背圧制御
Roadmap Update: Streams & HTTP
Already pretty mature and complete implementation.
WebSockets!
Play 2.5 (2.5+) uses Akka St...
• Reactive Platform
• Remoting / Cluster: Docker networking support
• Cluster: Split Brain Resolver (beta)
• Akka Persiste...
Links
• The projects:
• akka.io
• typesafe.com/products/typesafe-reactive-platform
• reactive-streams.org

• Viktor Klang’...
Thanks!
onNext(Q/A)
(Now’s the time to ask things!)
ktoso @ typesafe.com
twitter: ktosopl
github: ktoso
team blog: letitcr...
©Typesafe 2015 – All Rights Reserved
[Japanese] How Reactive Streams and Akka Streams change the JVM Ecosystem @ Reactive Shinjuku
Upcoming SlideShare
Loading in...5
×

[Japanese] How Reactive Streams and Akka Streams change the JVM Ecosystem @ Reactive Shinjuku

164
-1

Published on

Japanese subtitles by Yugo Maede-san, thank you very much. Japanese subtitled version of the "How Reactive Streams and Akka Streams change the JVM Ecosystem". http://www.slideshare.net/ktoso/how-reactive-streams-akka-streams-change-the-jvm-ecosystem

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
164
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
0
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

[Japanese] How Reactive Streams and Akka Streams change the JVM Ecosystem @ Reactive Shinjuku

  1. 1. Konrad 'ktoso' Malawski GeeCON 2014 @ Kraków, PL Konrad `@ktosopl` Malawski @ Reactive Systems Meetup streams How reactive streams change the JVM Ecosystem & reative streams と akka streams が JVMエコシステムをどのように変えるか
  2. 2. Konrad `ktoso` Malawski (we’re renaming soon!) Akka Team, Reactive Streams TCK, Maintaining Akka Http TypesafeのKonrad `ktoso` Malawskiです。 (Typesafeもうすぐ会社名を変更します)
  3. 3. Konrad `@ktosopl` Malawski akka.io typesafe.com geecon.org Java.pl / KrakowScala.pl sckrk.com / meetup.com/Paper-Cup @ London GDGKrakow.pl lambdakrk.pl (we’re renaming soon!)
  4. 4. Nice to meet you! Who are you guys? はじめまして!
  5. 5. Agenda for today: • Story & landscape • The Reactive Streams Protocol • Akka Streams / Demo • Akka Http / Demo • Q/A? 本日のアジェンダです。経緯と展望、Reacive Streams プロ トコル、Akka Streamsとデモ、Akka Http とデモ、Q/A
  6. 6. Reactive Streams - story: early FRP http://blogs.msdn.com/b/rxteam/archive/2009/11/17/announcing-reactive-extensions-rx-for-net-silverlight.aspx http://infoscience.epfl.ch/record/176887/files/DeprecatingObservers2012.pdf - Ingo Maier, Martin Odersky https://github.com/ReactiveX/RxJava/graphs/contributors https://github.com/reactor/reactor/graphs/contributors https://medium.com/@viktorklang/reactive-streams-1-0-0-interview-faaca2c00bec#.69st3rndy - .NETs’ Reactive Extensions .NET 3.5 Reactive Streams のストーリー: 初期のFRP .NET の Reactive Extensions がリリースされました。
  7. 7. Reactive Streams - story: 2013’s impls ~2013: Reactive Programming becoming widely adopted on JVM. - Play introduced “Iteratees” - Akka (2009) had Akka-IO (TCP etc.) - Ben starts work on RxJava http://blogs.msdn.com/b/rxteam/archive/2009/11/17/announcing-reactive-extensions-rx-for-net-silverlight.aspx http://infoscience.epfl.ch/record/176887/files/DeprecatingObservers2012.pdf - Ingo Maier, Martin Odersky https://github.com/ReactiveX/RxJava/graphs/contributors https://github.com/reactor/reactor/graphs/contributors https://medium.com/@viktorklang/reactive-streams-1-0-0-interview-faaca2c00bec#.69st3rndy Teams discuss need for back-pressure in simple user API. Play’s Iteratee / Akka’s NACK in IO. } 2013年には Reactive Programming が JVM 上で広く採用されるようになりました。 Play,Akkaのチームは簡潔なユーザAPIによる背圧制御の必要性を議論していました。
  8. 8. Reactive Streams - story: 2013’s impls Play Iteratees – pull back-pressure, difficult API http://blogs.msdn.com/b/rxteam/archive/2009/11/17/announcing-reactive-extensions-rx-for-net-silverlight.aspx http://infoscience.epfl.ch/record/176887/files/DeprecatingObservers2012.pdf - Ingo Maier, Martin Odersky https://github.com/ReactiveX/RxJava/graphs/contributors https://github.com/reactor/reactor/graphs/contributors https://medium.com/@viktorklang/reactive-streams-1-0-0-interview-faaca2c00bec#.69st3rndy Akka-IO – NACK back-pressure; low-level IO (Bytes); messaging API RxJava – no back-pressure, nice API Play の Iteratee は pull 型の背圧制御でAPIが難しく、Akka-IO は NACK 背圧制御で 低レベルI/OとメッセージングAPI, RxJava は背圧制御はないが優れたAPIでした。
  9. 9. Reactive Streams - Play’s Iteratees def fold[B]( done: (A, Input[E]) => Promise[B], cont: (Input[E] => Iteratee[E, A]) => Promise[B], error: (String, Input[E]) => Promise[B] ): Promise[B] // an iteratee that consumes chunkes of String and produces an Int Iteratee[String,Int] https://www.playframework.com/documentation/2.0/Iteratees Feb 2013 Iteratees solved the back-pressure problem, but were hard to use. Iteratee & Enumeratee – Haskell inspired. Play / Akka teams looking for common concept. 2013年2月, Play の Iteratee は背圧制御の問題を解決しましたが使いづらく、 Play/Akkaのチームは共通のコンセプトを探していました。
  10. 10. Reactive Streams - expert group founded October 2013 Roland Kuhn (Akka) and Erik Meijer (Rx .NET) meet in Lausanne, while recording “Principles of Reactive Programming” Coursera Course. Viktor Klang (Akka), Erik Meijer, Ben Christensen (RxJava) and Marius Eriksen (Twitter) meet at Twitter HQ. The term “reactive non-blocking asynchronous back-pressure” gets coined. 2013年10月, Roland Kuhn(Akka)、Erik Meijer (Rx .NET) らにより"リアクティブ ノンブロッキング非同期背圧制御"という言葉が造り出されました。
  11. 11. Reactive Streams - expert group founded October 2013 Roland Kuhn (Akka) and Erik Meijer (Rx .NET) meet in Lausanne, while recording “Principles of Reactive Programming” Coursera Course. Viktor Klang (Akka), Erik Meijer, Ben Christensen (RxJava) and Marius Eriksen (Twitter) meet at Twitter HQ. The term “reactive non-blocking asynchronous back-pressure” gets coined. Goals: - asynchronous - never block (waste) - safe (back-threads pressured) - purely local abstraction - allow synchronous impls. Also, for our examples today: - compatible with TCP 非同期、ブロックしない、安全、純粋にローカルのような抽象化、同期の実装も許すこ とをゴールとしました。また、今日の前例にならいTCPとの互換性も目指しました。
  12. 12. Reactive Streams - expert group founded October 2013 Roland Kuhn (Akka) and Erik Meijer (Rx .NET) meet in Lausanne, while recording “Principles of Reactive Programming” Coursera Course. Viktor Klang (Akka), Erik Meijer, Ben Christensen (RxJava) and Marius Eriksen (Twitter) meet at Twitter HQ. The term “reactive non-blocking asynchronous back-pressure” gets coined. December 2013 Stephane Maldini & Jon Brisbin (Pivotal Reactor) contacted by Viktor. Sephane Maldini と Jon Brisbin (Pival Reactor) がViktor にコン タクトしました。
  13. 13. Reactive Streams - expert group founded October 2013 Roland Kuhn (Akka) and Erik Meijer (Rx .NET) meet in Lausanne, while recording “Principles of Reactive Programming” Coursera Course. Viktor Klang (Akka), Erik Meijer, Ben Christensen (RxJava) and Marius Eriksen (Twitter) meet at Twitter HQ. The term “reactive non-blocking asynchronous back-pressure” gets coined. December 2013 Stephane Maldini & Jon Brisbin (Pivotal Reactor) contacted by Viktor. Soon after, the “Reactive Streams” expert group is formed. Also joining the efforts: Doug Lea (Oracle), EndreVarga (Akka), Johannes Rudolph & 
 Mathias Doenitz (Spray), and many others, including myself join the effort soon after. まもなく "Reactive Streams" エキスパートグループが創設されました。 Doug Lea(Oracle)や、私自身も含めた多くの人々が参画しました。
  14. 14. October 2013 Roland Kuhn (Akka) and Erik Meijer (Rx .NET) meet in Lausanne, while recording “Principles of Reactive Programming” Coursera Course. Viktor Klang (Akka), Erik Meijer, Ben Christensen (RxJava) and Marius Eriksen (Twitter) meet at Twitter HQ. The term “reactive non-blocking asynchronous back-pressure” gets coined. December 2013 Stephane Maldini & Jon Brisbin (Pivotal Reactor) contacted by Viktor. Soon after, the “Reactive Streams” expert group is formed. Also joining the efforts: Doug Lea (Oracle), EndreVarga (Akka), Johannes Rudolph & 
 Mathias Doenitz (Spray), and many others, including myself join the effort soon after. Reactive Streams - expert group founded I ended up implementing much of the TCK. Please use it, let me know if it needs improvements :-) 私はTCK(Technology Compatibility Kit) の大部分の実装を終えました。 ぜひ使ってみて、もし改良が必要なら教えてください :-)
  15. 15. Reactive Streams - story: 2013’s impls 2014–2015: Reactive Streams Spec & TCK development, and implementations. 1.0 released on April 28th 2015, with 5+ accompanying implementations. 2015 Proposed to be included with JDK9 by Doug Lea via JEP-266 “More Concurrency Updates” http://hg.openjdk.java.net/jdk9/jdk9/jdk/file/6e50b992bef4/src/java.base/share/classes/java/util/concurrent/Flow.java 2014-2015に Reactive Streams の仕様とTCKが開発・実装され、 2015年4月28日に 1.0 が5つ以上の実装と共にリリースされました。
  16. 16. 2014–2015: Reactive Streams Spec & TCK development, and implementations. 1.0 released on April 28th 2015, with 5+ accompanying implementations. 2015 Proposed to be included with JDK9 by Doug Lea via JEP-266 “More Concurrency Updates” http://hg.openjdk.java.net/jdk9/jdk9/jdk/file/6e50b992bef4/src/java.base/share/classes/java/util/concurrent/Flow.java Reactive Streams - story: 2013’s impls Doug Lea により、これを JDK9 に含めることが JEP-266 "More Concurrency Updates" を通じて提案されました。
  17. 17. in a few words: • Toolkit for building scalable distributed / concurrent apps. • High Performance Actor Model implementation • “share nothing” – messaging instead of sharing state • millions of msgs, per actor, per second • Supervision trees – built-in and mandatory • Clustering and Http built-in A B BarFoo C B E A D C /Foo /Foo/A /Foo/A/B /Foo/A/D Guardian System Actor Name resolution—like a file-system akka はスケーラブルな分散/並行アプリを構築するためのツールキットです。高性能な Actorモデルの実装で、Supervisionツリー、クラスタリング/Httpを組込んでいます。
  18. 18. Why back-pressure? ? なぜ背圧制御なのでしょうか?
  19. 19. Why back-pressure? So you’ve built your app and it’s awesome. 皆さんはアプリを構築してきました、 それは素晴らしいことです。
  20. 20. Why back-pressure? Let’s not smash it horribly under load. 負荷をかけすぎて、壊さないようにしましょう。
  21. 21. What is back-pressure? ? 背圧制御(back pressure) とは何でしょうか?
  22. 22. What is back-pressure? 背中(back)が痛くて死にそうだ!
  23. 23. No no no…! Not THAT Back-pressure! No no no…! Not THAT Back-pressure! What is back-pressure? いやいやいや...! その back pressure ではありません!
  24. 24. Publisher[T] Subscriber[T] Back-pressure explained 背圧制御の説明
  25. 25. Fast Publisher Slow Subscriber What if…? もしも、1秒間に100の命令を処理する高速なPublisherと、 1秒間に1つの命令しか処理しない低速なSuscriberだったら?
  26. 26. Push + NACK model Push と NACK モデル
  27. 27. Push + NACK model Subscriber usually has some kind of buffer. Subscriber は普通ある種のバッファを持っています。
  28. 28. Push + NACK model
  29. 29. Push + NACK model
  30. 30. What if the buffer overflows? Push + NACK model もしバッファがオーバーフローしたら何が起こるでしょう?
  31. 31. Use bounded buffer, drop messages + require re-sending Push + NACK model 大きさが限られたバッファを使い、 メッセージを廃棄し、再送を要求します。
  32. 32. Kernel does this! Routers do this! (TCP) Use bounded buffer, drop messages + require re-sending Push + NACK model カーネルがこれを行います! ルーターがこれを行います! (TCP)
  33. 33. Increase buffer size… Well, while you have memory available! Push + NACK model バッファサイズを増やしてみましょう... そう、使用できるメモリがある限りは!
  34. 34. Push + NACK model メモリ不足
  35. 35. Negative ACKnowledgement
  36. 36. NACKing Buffer overflow is imminent! バッファオーバーフローが今にも起こりそうです!
  37. 37. Telling the Publisher to slow down / stop sending… NACKing Publisher にスローダウンするか、送信を止めるよう知らせます...
  38. 38. NACK did not make it in time, because M was in-flight! NACKing NACK は間に合いませんでした、 メッセージが飛んできている最中だったからです!
  39. 39. What if… We don’t need to back-pressure, because: speed(publisher) < speed(subscriber) We need low-overhead for “happy case” もしも、publisher のスピード < subscrber のスピードであれば、 背圧制御は必要ありません。
  40. 40. No problem! Fast Subscriber => no problem 高速な Subscriber であれば、問題ありません!
  41. 41. Back-pressure? Reactive-Streams = “Dynamic Push/Pull” Fast Subscriber => no problem 背圧制御? Reactive Streams = "動的な Push/Pull"
  42. 42. Just push – not safe when Slow Subscriber Just pull – too slow when Fast Subscriber Reactive Streams: “dynamic push/pull” push だけだと、Subscriberが低速な時には、安全ではありません。 pull だけだと、Subscriberが高速な時には、遅すぎます。
  43. 43. Solution: Dynamic adjustment Just push – not safe when Slow Subscriber Just pull – too slow when Fast Subscriber Reactive Streams: “dynamic push/pull” 解決策は、動的な調整です。
  44. 44. Slow Subscriber sees it’s buffer can take 3 elements. Publisher will never blow up its buffer. Reactive Streams: “dynamic push/pull” 低速なSubscriberはバッファに3要素入れられることがわかります。 Publsherはバッファをあふれさせることは決してありません。
  45. 45. Fast Publisher will send at-most 3 elements. This is pull-based-backpressure. Reactive Streams: “dynamic push/pull” 高速なPublisherは、高々3つの要素を送ります。 これがpullベースの背圧制御です。
  46. 46. Fast Subscriber can issue more Request(n), before more data arrives! Reactive Streams: “dynamic push/pull” 高速なSubscriberは、データが到着する前に、 より多くのリクエストを発行することができます!
  47. 47. Fast Subscriber can issue more Request(n), before more data arrives. Publisher can accumulate demand. Reactive Streams: “dynamic push/pull” 高速なSubscriberは、需要を蓄積することができます。
  48. 48. Publisher accumulates total demand per subscriber. Reactive Streams: accumulate demand Publisherはsubscriberごとに需要の合計を蓄積します。
  49. 49. Total demand of elements is safe to publish. Subscriber’s buffer will not overflow. Reactive Streams: accumulate demand 需要の合計分の要素をpublishしても安全です。 Subscriberのバッファはオーバーフローしないでしょう。
  50. 50. http://reactive-streams.org We want to make different implementations co-operate with each other. Reactive Streams: Inter Op 私たちは、異なる実装がお互いに協力できるようにしたいのです。
  51. 51. http://reactive-streams.org We want to make different implementations co-operate with each other. Reactive Streams: Inter Op 私たちは、異なる実装がお互いに協力できるようにしたいのです。
  52. 52. RS is NOT a “daily use”, “end-user” API. It’s an SPI - Service Provider Interface. Reactive Streams: Inter-Op https://en.wikipedia.org/wiki/Service_provider_interface Service Provider Interface (SPI) is an API intended to be implemented or extended by a third party. RS は "毎日使用する", "エンドユーザ向けの" API ではなく、サードパーティにより 実装される、または拡張される SPI (Service Provider Interface) です。
  53. 53. EmbeddedApp.fromHandler(new Handler { override def handle(ctx: Context): Unit = { // RxJava Observable val intObs = Observable.from((1 to 10).asJava) // Reactive Streams Publisher val intPub = RxReactiveStreams.toPublisher(intObs) // Akka Streams Source val stringSource = Source(intPub).map(_.toString) // Reactive Streams Publisher val stringPub = stringSource.runWith(Sink.fanoutPublisher(1, 1)) // Reactor Stream val linesStream = Streams.create(stringPub).map[String]( new reactor.function.Function[String, String] { override def apply(in: String) = in + "n" }) // and now render the HTTP response (RatPack) ctx.render(ResponseChunks.stringChunks(linesStream)) } }).test(new Consumer[TestHttpClient] { Reactive Streams: Inter-Op https://en.wikipedia.org/wiki/Service_provider_interface
  54. 54. EmbeddedApp.fromHandler(new Handler { override def handle(ctx: Context): Unit = { // RxJava Observable val intObs = Observable.from((1 to 10).asJava) // Reactive Streams Publisher val intPub = RxReactiveStreams.toPublisher(intObs) // Akka Streams Source val stringSource = Source(intPub).map(_.toString) // Reactive Streams Publisher val stringPub = stringSource.runWith(Sink.fanoutPublisher(1, 1)) // Reactor Stream val linesStream = Streams.create(stringPub).map[String]( new reactor.function.Function[String, String] { override def apply(in: String) = in + "n" }) // and now render the HTTP response (RatPack) ctx.render(ResponseChunks.stringChunks(linesStream)) } }).test(new Consumer[TestHttpClient] { Reactive Streams: Inter-Op https://en.wikipedia.org/wiki/Service_provider_interface
  55. 55. Akka Streams streams
  56. 56. Akka Streams & HTTP streams & HTTP
  57. 57. Akka Streams in 20 seconds: // types: Source[Out, Mat] Flow[In, Out, Mat] Sink[In, Mat] // generally speaking, it's always: val ready = Source(???).via(flow).map(_ * 2).to(sink) val mat: Mat = ready.run() // the usual example: val f: Future[String] = Source.single(1).map(_.toString).runWith(Sink.head) Proper static typing! Akka Streams を 20秒で説明します。 Source, Flow, Sink に適切な静的型付けをします。
  58. 58. Akka Streams in 20 seconds: // types: _ Source[Int, akka.NotUsed] Flow[Int, String, akka.NotUsed] Sink[String, Future[String]] Source.single(1).map(_.toString).runWith(Sink.head)
  59. 59. Akka Streams in 20 seconds: // types: _ Source[Int, Unit] Flow[Int, String, Unit] Sink[String, Future[String]] Source.single(1).map(_.toString).runWith(Sink.head)
  60. 60. Akka HTTP Joint effort of Spray and Akka teams. Complete HTTP Server/Client implementation. Soon prod ready, developed ~1.5 years. Learns from Spray’s 3-4 years history. Since the beginning with streaming as first class citizen. Akka HTTP は、SprayとAkkaチームが協力して開発した、完全なHTTPサーバ/クライアント 実装です。1年半かけて開発されており、間もなく業務に耐えうるレベルになるでしょう。
  61. 61. It’s turtles buffers all the way down! 親亀の上に子亀、子亀の上に孫亀がのるかのように、バッファの 次にバッファ、その次にまたバッファ... と無数につらなります!
  62. 62. Streaming from Akka HTTP Akka HTTP からのストリーミング
  63. 63. Streaming from Akka HTTP Akka HTTP からのストリーミング
  64. 64. Streaming from Akka HTTP
  65. 65. Streaming from Akka HTTP
  66. 66. Streaming from Akka HTTP No demand from TCP = No demand upstream = Source won’t generate tweets TCPからの要求がない = 上流の要求がない = Source はツイートを生成しません
  67. 67. Streaming from Akka HTTP No demand from TCP = No demand upstream = Source won’t generate tweets => TCPからの要求がない = 上流の要求がない = Source はツイートを生成しません
  68. 68. Streaming from Akka HTTP No demand from TCP = No demand upstream = Source won’t generate tweets => Bounded memory stream processing! =>大きさが制限されたメモリーでのストリーム処理!
  69. 69. Client / Server “JSON Streaming” demo Demo time クライアント/サーバの "JSONストリーミング" デモ
  70. 70. Akka Streams Hidden powers: Parallelism && Pipelining Akka Streams の隠されたパワー: 並列処理(Parallelism)とパイプライン
  71. 71. Pipelining Pancakes http://doc.akka.io/docs/akka-stream-and-http-experimental/1.0/scala/stream-parallelism.html
  72. 72. Pipelining
  73. 73. Pipelining
  74. 74. Pipelining // Takes a scoop of batter and creates a pancake with one side cooked val fryingPan1: Flow[ScoopOfBatter, HalfCookedPancake, Unit] = Flow[ScoopOfBatter].map { batter => HalfCookedPancake() } // Finishes a half-cooked pancake val fryingPan2: Flow[HalfCookedPancake, Pancake, Unit] = Flow[HalfCookedPancake].map { halfCooked => Pancake() } // With the two frying pans we can fully cook pancakes val pancakeChef: Flow[ScoopOfBatter, Pancake, Unit] = Flow[ScoopOfBatter].via(fryingPan1).via(fryingPan2)
  75. 75. Parallelism ???
  76. 76. Parallelism
  77. 77. Parallelism val fryingPan: Flow[ScoopOfBatter, Pancake, Unit] = Flow[ScoopOfBatter].map { batter => Pancake() } val pancakeChef: Flow[ScoopOfBatter, Pancake, Unit] = Flow() { implicit builder => val dispatchBatter = builder.add(Balance[ScoopOfBatter](2)) val mergePancakes = builder.add(Merge[Pancake](2)) dispatchBatter.out(0) ~> fryingPan ~> mergePancakes.in(0) dispatchBatter.out(1) ~> fryingPan ~> mergePancakes.in(1) (dispatchBatter.in, mergePancakes.out) }
  78. 78. Parallelism val fryingPan: Flow[ScoopOfBatter, Pancake, Unit] = Flow[ScoopOfBatter].map { batter => Pancake() } val pancakeChef: Flow[ScoopOfBatter, Pancake, Unit] = Flow() { implicit builder => val dispatchBatter = builder.add(Balance[ScoopOfBatter](2)) val mergePancakes = builder.add(Merge[Pancake](2)) dispatchBatter.out(0) ~> fryingPan ~> mergePancakes.in(0) dispatchBatter.out(1) ~> fryingPan ~> mergePancakes.in(1) (dispatchBatter.in, mergePancakes.out) }
  79. 79. Parallelism val fryingPanFun: ScoopOfBatter ⇒ Future[Pancake] = batter ⇒ Future.successful(Pancake()) val pancakeChef: Flow[ScoopOfBatter, Pancake, Unit] = Flow[ScoopOfBatter].mapAsync(parallelism = 2)(fryingPanFun) Or simply “mapAsync”:
  80. 80. Pipelining && Parallelism Parallelism && Pipelining do the heavy-work for you. 並列処理とパイプラインは、あなたのために きつい作業をやってくれます。
  81. 81. 10/26/2015 spray-can: add websockets support (client & server) · Issue #134 · spray/spray Pull requests Issues GistThis repository Search 2,092 496197Watch Star Forkspray / spray and others Labels Milestone   akka-http Assignee No one assigned 111 participants spray-can: add websockets support (client & server) #134 Closed sirthias opened this issue on Sep 4, 2012 · 129 comments New issue Feature Notifications You’re not receiving notifications from this thread. Subscribe Ownersirthias commented on Sep 4, 2012 No description provided. analytically commented on Oct 23, 2012 +1 tommcp commented on Nov 1, 2012 +1 t3hnar commented on Nov 10, 2012 +1 alexbool commented on Nov 10, 2012 +1 olger commented on Nov 16, 2012 +1 pjean commented on Nov 29, 2012 +1 edgurgel commented on Nov 29, 2012 +1 zerni commented on Dec 10, 2012 +1 Bathtor commented on Dec 10, 2012 +1 WebSockets A.K.A. “Spray’s single most upvoted feature request ever” 98 * “+1” WebSockets は、"Spray のこれまでの機能リクエストのなかで、 単独で最も票を集めた機能" としても知られています。
  82. 82. Spray’s most requested feature ever: WebSockets path("ws") { val handler: Flow[Message, Message] = ??? handleWebsocketMessages(handler) }
  83. 83. Spray’s most requested feature ever: WebSockets path("ws") { val handler: Flow[Message, Message] = ??? handleWebsocketMessages(handler) }
  84. 84. Spray’s most requested feature ever: WebSockets path("ws") { val handler = Flow.fromSinkAndSource( Sink.ignore, Source.single(TextMessage("Hello World!”))) handleWebsocketMessages(handler) }
  85. 85. Summing up… Summing up… 要約...
  86. 86. buffers, buffers everywhere! https://dev.twitter.com/streaming/overview/request-parameters#stallwarnings あらゆるところにバッファが!
  87. 87. JEP-266 – soon…! public final class Flow { private Flow() {} // uninstantiable @FunctionalInterface public static interface Publisher<T> { public void subscribe(Subscriber<? super T> subscriber); } public static interface Subscriber<T> { public void onSubscribe(Subscription subscription); public void onNext(T item); public void onError(Throwable throwable); public void onComplete(); } public static interface Subscription { public void request(long n); public void cancel(); } public static interface Processor<T,R> extends Subscriber<T>, Publisher<R> { } } JEP-266 がまもなく...!
  88. 88. Back-pressure as a feature 特徴(目玉機能)としての背圧制御
  89. 89. Roadmap Update: Streams & HTTP Already pretty mature and complete implementation. WebSockets! Play 2.5 (2.5+) uses Akka Streams. (Scala || Java) DSL == same power. Last phases of polishing up APIs and features. 2.4.2 release in coming weeks. Akka 2.4 requires JDK8. (that’s about time to do so!) StreamsとHTTPの最新ロードマップ 既に十分成熟しており、実装が完了しました。
  90. 90. • Reactive Platform • Remoting / Cluster: Docker networking support • Cluster: Split Brain Resolver (beta) • Akka Persistence: Cross-Scala-version snapshot deserializer • Java 6: Extended LTS • Akka 2.4.2 (released this month, binary compatible with 2.3) • Akka Streams becomes non experimental in 2.4.2! • Cluster Tools promoted to stable! • Persistence promoted to stable! • Persistence Queries (experimental) • Akka Typed (experimental) • Distributed Data (experimental) Roadmap Update: Akka Akka の最新ロードマップ
  91. 91. Links • The projects: • akka.io • typesafe.com/products/typesafe-reactive-platform • reactive-streams.org
 • Viktor Klang’s interview with all RS founding members • Akka HTTP in depth with Mathias and Johannes @ Scala.World • Akka User - mailing list: • https://groups.google.com/group/akka-user • Community chat: • http://gitter.im/akka/akka
  92. 92. Thanks! onNext(Q/A) (Now’s the time to ask things!) ktoso @ typesafe.com twitter: ktosopl github: ktoso team blog: letitcrash.com home: akka.io
  93. 93. ©Typesafe 2015 – All Rights Reserved
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×