Scala Days NYC 2016

Scala Days NYC 2016

1. 1. Scala The Road Ahead Martin Odersky Scala Days 2016, NYC
2. 2. 2015
3. 3. 2015 was on the quiet side •  Maturing tools: 2.11.x, IDEs, sbt •  Steady growth indeed.com jobs google trends
4. 4. 2016
5. 5. In 2016, things are heating up again •  The Scala Center •  Scala 2.12 release •  Rethinking the Scala libraries •  New target platforms •  DOT and dotty
6. 6. A New Steward for Scala -  Undertake projects that benefit all of the Scala community. -  Industrial support by Lightbend, Goldman Sachs, Nitro, IBM, 47 Degrees, Verizon. -  More in Heather Miller’s Keynote tomorrow.
7. 7. Scala 2.12 Optimized for Java 8 •  Uses Java 8’s lambdas and default methods •  for shorter code and faster execution speed.
 Projected release date: mid 2016. In case you are still on Java 6/7, Scala 2.11 will be around for a while.
8. 8. 33 features in release notes
9. 9. 2.12 Data 33 features in release notes 336 PRs closed (40 more to go) 65 committers 177 Lukas Rytz 76 Jason Zaugg 70 Adriaan Moors 33 A. P. Marki 25 Felix Mulder 17 Simon Ochsenreither 14 Janek Bogucki 14 Rex Kerr 12 Sébastien Doeraene
10. 10. And, finally: •  Programming in Scala
 3rd edition is out. •  Updated for 2.12. •  Book signing 
 Tuesday lunch 
 12.20-13.20 at
 the Artima booth.
11. 11. Beyond 2.12 Scala 2.13 will focus on the libraries Plans to revamp collections •  Even simpler to use •  More in line with Spark usage •  Better lazy collections (views) Current Status: •  Strawman proposals under study. •  Would like to have more of them,
12. 12. Beyond 2.12 Scala 2.13 will focus on the libraries Better modularization. We are thinking about a split:
 Scala core Scala stdlib Scala platform Your input and help is vital for this!
13. 13. Scala.js 0.6.9 •  “native” anonymus classes •  js.TupleN •  JUnit support •  Faster code generation
14. 14. COMING SOO
15. 15. And then there is the
16. 16. DOT We finally have a proven foundation for Scala! The DOT calculus talks about a minimal language subset, chosen so that •  we can make and prove formal statements about it •  we can encode much of the rest of the language in it. This concludes an 8 year effort! It opens the door to do language work with much better confidence than before.
17. 17. DOT Terms •  Translated to Scala notation, the language covered by DOT is: Value v = (x: T) => t Function new { x: T => d } Object Definition d = def a = t Method definition
 type A = T Type Term t = v Value
 x Variable
 t1(t2) Application
 t.a Selection
 { val x = t1; t2 } Local definition.
18. 18. DOT Types The Types covered by DOT are: Type T = Any Top type
 Nothing Bottom type
 x.A Selection
 (x: T1) => T2 Function
 { def a: T } Method declaration
 { type T >: T1 <: T2 } Type declaration
 T1 & T2 Intersection
 { x => T } Recursion
19. 19. Type Soundness The following property was shown with a mechanized proof: If a term t has type T,
 and evaluation of t terminates: the result will be a value v of type T. Why is this Important? It gives us a technique to reason about correctness of other language features.
20. 20. dotty dotty is working name for our new Scala compiler. •  Builds on DOT in its internal data structures. •  Generics get expressed as type members. •  Supports an evolution of the Scala programming language. •  A first developer preview is around the corner •  Targeted at contributors and experimenters.
21. 21. dotty – Technical Data A bit more than half the size of the current Scala compiler, nsc. •  dotty: 45 KLoc •  nsc: 75 KLoc •  About twice the speed of nsc. •  should improve significantly in the future.
22. 22. dotty Architecture dotty Frontend dotty Transforms GenBCode Classfiles Simplified AST AST Scala Sources nsc Scala Sources TASTY Pickled nsc Transforms Simplified AST AST
23. 23. Evolving The Language My Goal: Make Scala “best” programming language 
 I know to how make. There is not agreement on what is “best”, so all we can really do is try to find a local optimum. But finding that optimium is what drives me.
24. 24. Essential Elements Conceptually we play with: •  Functions •  Classes and objects, •  Strict evaluation. •  Local type inference •  Implicits. Many languages are currently moving in that space
25. 25. Goals 1.  Deepen the synthesis of functional programming and modular programming. 2.  Improve the connection of Scala with its theoretical foundations. 3.  Improve the guarantees of type system. 4.  But stay simple and approachable.
26. 26. A Whirlwind Tour Of Dotty
27. 27. Dropped Features DelayedInit Macros Existential Types Procedure Syntax Early Initializers General Type 
 Projection
28. 28. Dropped Features DelayedInit Macros Existential Types Procedure Syntax Early Initializers General Type 
 Projection def run() { ... } Will be rewritten automatically to def run(): Unit = { ... }
29. 29. Dropped Features DelayedInit Macros Existential Types Procedure Syntax Early Initializers General Type 
 Projection class Foo extends DelayedInit
30. 30. Dropped Features DelayedInit Macros Existential Types Procedure Syntax Early Initializers General Type 
 Projection (the reflection 
 based kind) def m(...) = macro impl(...)
31. 31. Dropped Features DelayedInit Macros Existential Types Procedure Syntax Early Initializers General Type 
 Projection class C extends { val x = e } with D Use trait parameters instead
32. 32. Dropped Features DelayedInit Macros Existential Types Procedure Syntax Early Initializers General Type 
 Projection C[U] forSome { type U } Wildcards C[_]still supported.
33. 33. Dropped Features DelayedInit Macros Existential Types Procedure Syntax Early Initializers General Type 
 Projection T # X - Was shown to be unsound for general types T. - Projection C#X from class types C still available.
34. 34. Implemented New Features Multiversal Equality Intersection Types Union types Trait parameters Function arity adaptation Named type parameters @static methods and fields non-blocking lazy vals
35. 35. Implemented New Features Multiversal Equality Intersection Types Union types Trait parameters Function arity adaptation Named type parameters T & U - replaces T with U - is commutative @static methods and fields non-blocking lazy vals
36. 36. Implemented New Features Multiversal Equality Intersection Types Union types Trait parameters Named type parameters T | U avoids huge lubs @static methods and fields non-blocking lazy vals
37. 37. Implemented New Features Multiversal Equality Intersection Types Union types Trait parameters Function arity adaptation Named type parameters pairs.map((x, y) => x + y) 
 instead of pairs.map { case (x, y) => x + y } @static methods and fields non-blocking lazy vals
38. 38. Implemented New Features Multiversal Equality Intersection Types Union types Trait parameters Function arity adaptation Named type parameters @static methods and fields non-blocking lazy vals trait T(x: Int) { ... }
39. 39. Implemented New Features Multiversal Equality Intersection Types Union types Trait parameters Function arity adaptation Named type parameters @static methods and fields non-blocking lazy vals object O { @static val x = ... @static def f() = ... }
40. 40. Implemented New Features Multiversal Equality Intersection Types Union types Trait parameters Function arity adaptation Named type parameters @static methods and fields non-blocking lazy vals lazy val x = ... // thread-local @volatile lazy val x - ... // thread-safe, 
 // avoids dead-locks
41. 41. Implemented New Features Multiversal Equality Intersection Types Union types Trait parameters Function arity adaptation Named type parameters @static methods and fields non-blocking lazy vals type-safe ==, !=
42. 42. Implemented New Features Multiversal Equality Intersection Types Union types Trait parameters Function arity adaptation Named type parameters trait Map[type Key, type Value] Map[Key = Int] allows partial type parameterization: - specify some types, - leave others to be inferred @static methods and fields non-blocking lazy vals
43. 43. Implemented New Features Multiversal Equality Intersection Types Union types Trait parameters Function arity adaptation Named type parameters Motivation •  Better foundations •  Easier, safer to use •  More orthogonal @static methods and fields non-blocking lazy vals
44. 44. Trait or Class?
 Classes and traits now have largely the same capabilities. Rule of thumb: •  When it's fully defined, make it a class •  When it's abstract, make it a trait Abstract classes are retained mainly for Java interop and for optimization.
45. 45. Improvements in Detail •  Type System •  Strongly influenced by DOT •  Better integration of type refinements •  Type Inference •  Subtyping constraint solver •  Inference is simpler to specify •  Implicit Search •  Faster search algorithm. •  Better behaved for contravariant types. •  Value Classes •  Nested value classes •  Arrays of value classes.
46. 46. Who’s working on all this? •  So far, mostly team at EPFL, including •  Dmitry Petrashko @darkdimius •  Guillaume Martres @smarter •  Vladimir Nikolayev @vladimirnik •  Felix Mulder @felixmulder •  Scala team at Lightbend helps with infrastructure, reviews and suggestions. •  Your contribution would be very much appreciated! •  Without it, nothing much will happen.
47. 47. Advances in Tooling •  SBT integration •  Basic integration exists •  We are working on incremental compilation. •  REPL •  With syntax highlighting! •  IDEs •  Jetbrains working on IntelliJ plugin. •  Doc generation •  Dynamic hyperlinks for cross-linking libraries. •  Linker •  A whole-program analyzer/opimizer •  Uses TASTY for serialization •  Makes specialization cheaper and more robust
48. 48. And the Future?
49. 49. Planned In Future Releases scala.meta Generic Programming Better records Implicit Function Types null safety Effect System
50. 50. Planned In Future Releases scala.meta Generic Programming Better records Implicit Function Types null safety Effect System inline def m(inline x: Int, y: Float): Float = meta { ... } - inline for inlining, meta for meta-programming. - run by an interpreter (no reflection) - meta uses quasi quotes for matching and construction - blackbox and annotation macros Why the change? - Simpler - Fewer implementation dependencies - Safer, since interpretation allows sandboxing - Restrict syntactic freedom, since no whitebox macros.
51. 51. Planned In Future Releases scala.meta Generic Programming Better records Implicit Function Types null safety Effect System type CtxS = implicit Context => S def f(x: T): CtxS = { ... implicitly[Context] ... } f(e) // Context argument is passed implicitly Why the change? - Small as it looks, this is huge - Allows abstraction over implicit parameters - Eliminates boilerplate
52. 52. Planned In Future Releases scala.meta Generic Programming Better records Implicit Function Types null safety Effect System Effects are implicit capabilities Two function arrows: A => B (impure) A -> B (pure) Why the change? - Effect checking is very much in
 demand. - I believe we can do better than
 monads to achieve it.
 - Implicits are a natural fit.
53. 53. Planned In Future Releases scala.meta Generic Programming Better records Implicit Function Types null safety Effect System Model nullable types as a union types T? = T | Null - Types don’t have null by default - Values coming from Java get a “?”. E.g. 
 System.out: PrintStream? - null dereferencing is an effect. - System.out.println() is ok in impure code. Why the change? - null poses unsoundness problems. - Null-safety is conceptually easy once you have
 union types and effects.
54. 54. Planned In Future Releases scala.meta Generic Programming Better records Implicit Function Types null safety Effect System “Scrap Your Boilerplate” - Product-of-sum interpretation of case class hierarchies - Tuples are HLists, but implemented more efficiently (S, T, U) = (S, (T, (U, Unit))) - Tuple22/Product22/Function22 will be history. Why the change? - Abstract over arity. - Define behavior for ADTs in a generic way
55. 55. Planned In Future Releases scala.meta Generic Programming Better records Implicit Function Types null safety Effect System - Analogue of tuples, but with labels. - Should integrate with generic programming - Should be implementable by a HashMap instead of by reflection. Why the change? - Existing structural types are not very useful. - Need something more flexible, and hopefully simpler.
56. 56. What About Guard Rails? •  Scala’s premise: Trust devs to do the right thing. •  But what if they don’t? •  And can we even agree what the right thing is?
57. 57. First Advice: Read this
58. 58. Containment Problem: Even if our code base uses the principle of least power, we do not know whether the libraries that we depend on do the same. E.g. library X might define an implicit conversion implicit def i2s(x: Int): String (patterns like this are more common in practice than you’d like to think) So instead of least power you get maximal surprise.
59. 59. A Modest Proposal For a conversion implicit def c(x: A): B make it a style error if the conversion -  is not defined in one of the packages containing A or B. -  is itself public (visible outside its package). When using a library defining such conversions, warn on the client side as well.
60. 60. Flexibility: Blessing or Curse? Scala has great syntactic flexibility. Sometimes this can be a burden. Example: Should I write xs.map(f) or xs map f ? What about xs.map(f).filter(p) vs xs map f filter p x.min(y) vs x min y ?
61. 61. Another Modest Proposal •  Add an annotation @infix that indicates that an operator is supposed to be used infix. @infix def min(y: Int): Int •  Make it a style error if an operator is used in the 
 wrong mode. x min y // ok xs map f // warning: Infix usage of normal method map
62. 62. Symbolic Operators •  If the operator is symbolic, require in addition an alphanumeric alias: @infix(“append”) def += (elem: T) •  Make it a style error if the alias is missing à One knows how to pronounce the operator à It’s easier to search for it, e.g. 
 “Scala append operator” as a search term. •  To be worked out: What are the precise rules for style errors?
63. 63. Thank You