Modern SQL in PostgreSQL

1. Still using Windows 3.1? So why stick to SQL-92? Modern SQL in PostgreSQL @MarkusWinand

2. SQL:1999

3. !"#$%"!

4. Inline views can't refer to outside the view: !"#"$%&' &&()*+&,- &&.*/0&1!"#"$%&' &&&&&&&&&&()*+&,2 &&&&&&&&&34")"&,256&7&,-56 &&&&&&&8&9:;9:<=>9<? &&&&*0&19:;9:<=>9<?56&7&,-568 LATERAL Before SQL:1999 Invalid

5. Inline views can't refer to outside the view: !"#"$%&' &&()*+&,- &&.*/0&1!"#"$%&' &&&&&&&&&&()*+&,2 &&&&&&&&&34")"&,256&7&,-56 &&&&&&&8&9:;9:<=>9<? &&&&*0&19:;9:<=>9<?56&7&,-568 LATERAL Before SQL:1999 Belongs there

6. SQL:99 #@%")@# views can: !"#"$%&' &&()*+&,- &&.*/0&!"#$%"!&1!"#"$%&' &&&&&&&&&&&&&&&&&&()*+&,2 &&&&&&&&&&&&&&&&&34")"&,256&7&,-56 &&&&&&&&&&&&&&&8&9:;9:<=>9<? &&&&*0&1,AB<8 LATERAL Since SQL:1999 Valid due to LATERAL keyword Useless, but still required except for CROSS join

7. But WHY?

8. Apply #/+/% per row from previous table: !"#"$%&,CD=DACEBF,G5' &&()*+&FH,<ICA9<G&F &&.*/0&#@%")@#&1!"#"$%&' &&&&&&&&&&&&&&&&&&()*+&DACEBF,G&D &&&&&&&&&&&&&&&&&34")"&D5FH,&7&F5FH, &&&&&&&&&&&&&&&&&'%($%&)*&+,-./0&($12 &&&&&&&&&&&&&&&&&!343#&5 &&&&&&&&&&&&&&&8&,CD=DACEBF,G LATERAL and Top-N per Group

9. Get the 10 most recent news for subscribed topics: !"#"$%&:5' &&()*+&:<?G&: &&.*/0&GBJGFA9D,9C:G&G &&&&*0&1:5,CD9F&7&G5,CD9F8 &34")"&G5BG<A&7&K &*)L")&MN&:5FA<H,<E&L"!$ &#/+/%&-O LATERAL and Multi-Source Top-N

10. LATERAL and Multi-Source Top-N Sort/Reduce Join everything !"#"$%&$"#'()*+ ,-,%./01(2-3 45%6/.$%&$"#'()*+ ,-,%./01(2-3 %%%6/.$%7'$8/9:%$/;4<%8'=;1/.$%7'#:%*->? %%%45%@=18%A/"B%&$"#'()** C--%./01(D-E -)D3 %%%%%%45%6'F%6G=B%/B%1HI1G.";$"/B1%1 %%%%%%%%%&$"#'(*+D%./01(C-3 %%%%%%45%@=18%&$"#'(2-J DC+%./01(2-K,3 %%%%%%%%%45%6'F%6G=B%/B%B'01%B %%%%%%%%%%%%&$"#'(D2 )2C%./01(2-K,3 LM=BB"BN%$"#':%-O)DJ%#1 PQ'GH$"/B%$"#':%)*+ ,-,O)+2%#1

11. LATERAL and Multi-Source Top-N Sort/Reduce Join everything #9P9,&1,9P<72QR SOS&AC?G7-O8 TU&!CA,&1,9P<72QR SOS&AC?G7-O8 &&&!CA,&+<,VCEW&,CDT0&V<HDGCA,&+<PW&QOXM &&&TU&4HGV&.C9:&1,9P<72QQ YOO&AC?G7ZO[ O2Z8 &&&&&&TU&!<&!FH:&C:&GBJGFA9D,9C:G&G &&&&&&&&&1,9P<7QRZ&AC?G7YO8 &&&&&&TU&4HGV&1,9P<7-O] ZYR&AC?G7-O^S8 &&&&&&&&&TU&!<&!FH:&C:&:<?G&: &&&&&&&&&&&&1,9P<7Z- 2-Y&AC?G7-O^S8 _;H::9:I&,9P<W&O52Z]&PG "6<FB,9C:&,9P<W&2QR SOS52R-&PG Why producing 900k rows... ...when there are only 80 subscriptions?

12. LATERAL and Multi-Source Top-N Sort/Reduce Join everything #9P9,&1,9P<72QR SOS&AC?G7-O8 TU&!CA,&1,9P<72QR SOS&AC?G7-O8 &&&!CA,&+<,VCEW&,CDT0&V<HDGCA,&+<PW&QOXM &&&TU&4HGV&.C9:&1,9P<72QQ YOO&AC?G7ZO[ O2Z8 &&&&&&TU&!<&!FH:&C:&GBJGFA9D,9C:G&G &&&&&&&&&1,9P<7QRZ&AC?G7YO8 &&&&&&TU&4HGV&1,9P<7-O] ZYR&AC?G7-O^S8 &&&&&&&&&TU&!<&!FH:&C:&:<?G&: &&&&&&&&&&&&1,9P<7Z- 2-Y&AC?G7-O^S8 _;H::9:I&,9P<W&O52Z]&PG "6<FB,9C:&,9P<W&2QR SOS52R-&PG Only the 10 most recent per subscription, you need.

13. !"#"$%&:5' &&()*+&GBJGFA9D,9C:G&G &&.*/0&#@%")@#&1!"#"$%&' &&&&&&&&&&&&&&&&&&()*+&:<?G&: &&&&&&&&&&&&&&&&&34")"&:5,CD9F&7&G5,CD9F &&&&&&&&&&&&&&&&&'%($%&)*&/,6-7.879&($12 &&&&&&&&&&&&&&&&&!343#&:; &&&&&&&&&&&&&&&8&,CD=:<?G&*0&1,AB<8 &34")"&G5BG<A=9E&7&K &*)L")&MN&:5FA<H,<E&L"!$ &#/+/%&-O LATERAL and Multi-Source Top-N

14. #9P9,&1,9P<725]YY&AC?G7-O8 TU&!CA,&1,9P<725]YS&AC?G7-O8 &&&TU&0<G,<E&#CCD&1,9P<725QQZ&-<=>?@;;8 &&&&&&TU&/:E<6&*:;`&!FH:&BG9:I&DX&C:&G&&& &&&&&&&&&1,9P<7O5O]2&AC?G7YO8 &&&&&&TU&#9P9, &&&&&&&&&1,9P<7O5O2S&AC?G7-O&;CCDG7YO8 &&&&&&&&&TU&/:E<6&!FH:&MHFX?HAE &&&&&&&&&&&&BG9:I&:<?G=,CD9F=,G=9E&C:&:&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&& _;H::9:I&,9P<W&O5-R-&PG "6<FB,9C:&,9P<W&25[-Z&PG LATERAL and Multi-Source Top-N About 100 000 times faster Limited to 10 times # of subscriptions

15. LATERAL in an Nutshell #@%")@# is the "for each" loop of SQL #@%")@# plays well with outer joins #@%")@# is an optimization Super-Power #@%")@# handy to join table functions

16. LATERAL Availability (SQL:1999)

17. &'#( (Common Table Expressions)

18. WITH Before SQL:99 Nested queries are hard to read: !"#"$%&a &&()*+&1!"#"$%&a &&&&&&&&&&()*+&,- &&&&&&&&&&.*/0&1!"#"$%&a&()*+&a &&&&&&&&&&&&&&&8&H&*0&1a8 &&&&&&&8&J &&.*/0&1!"#"$%&a&()*+&a &&&&&&&8&F&*0&1a8

19. Understand this first WITH Before SQL:99 Nested queries are hard to read: !"#"$%&a &&()*+&1!"#"$%&a &&&&&&&&&&()*+&,- &&&&&&&&&&.*/0&1!"#"$%&a&()*+&a &&&&&&&&&&&&&&&8&H&*0&1a8 &&&&&&&8&J &&.*/0&1!"#"$%&a&()*+&a &&&&&&&8&F&*0&1a8

20. WITH Before SQL:99 Then this... Nested queries are hard to read: !"#"$%&a &&()*+&1!"#"$%&a &&&&&&&&&&()*+&,- &&&&&&&&&&.*/0&1!"#"$%&a&()*+&a &&&&&&&&&&&&&&&8&H&*0&1a8 &&&&&&&8&J &&.*/0&1!"#"$%&a&()*+&a &&&&&&&8&F&*0&1a8

21. WITH Before SQL:99 Then this... Nested queries are hard to read: !"#"$%&a &&()*+&1!"#"$%&a &&&&&&&&&&()*+&,- &&&&&&&&&&.*/0&1!"#"$%&a&()*+&a &&&&&&&&&&&&&&&8&H&*0&1a8 &&&&&&&8&J &&.*/0&1!"#"$%&a&()*+&a &&&&&&&8&F&*0&1a8

22. WITH Before SQL:99 Finally the first line makes sense Nested queries are hard to read: !"#"$%&a &&()*+&1!"#"$%&a &&&&&&&&&&()*+&,- &&&&&&&&&&.*/0&1!"#"$%&a&()*+&a &&&&&&&&&&&&&&&8&H&*0&1a8 &&&&&&&8&J &&.*/0&1!"#"$%&a&()*+&a &&&&&&&8&F&*0&1a8

23. CTEs are statement-scoped views: 3/%4 &H&1F-b&F2b&FQ8 @!&1!"#"$%&F-b&F2b&FQ&()*+&a8b &J&1F]b&a8 @!&1!"#"$%&F]b&a &&&&&&()*+&,- &&&&&&.*/0&H &&&&&&&&*0&1a8 &&&8b WITH Since SQL:99

24. CTEs are statement-scoped views: 3/%4 &H&1F-b&F2b&FQ8 @!&1!"#"$%&F-b&F2b&FQ&()*+&a8b &J&1F]b&a8 @!&1!"#"$%&F]b&a &&&&&&()*+&,- &&&&&&.*/0&H &&&&&&&&*0&1a8 &&&8b Keyword WITH Since SQL:99

25. CTEs are statement-scoped views: 3/%4 &H&1F-b&F2b&FQ8 @!&1!"#"$%&F-b&F2b&FQ&()*+&a8b &J&1F]b&a8 @!&1!"#"$%&F]b&a &&&&&&()*+&,- &&&&&&.*/0&H &&&&&&&&*0&1a8 &&&8b WITH Since SQL:99 Name of CTE and (here optional) column names

26. CTEs are statement-scoped views: 3/%4 &H&1F-b&F2b&FQ8 @!&1!"#"$%&F-b&F2b&FQ&()*+&a8b &J&1F]b&a8 @!&1!"#"$%&F]b&a &&&&&&()*+&,- &&&&&&.*/0&H &&&&&&&&*0&1a8 &&&8b WITH Since SQL:99 Definition

27. CTEs are statement-scoped views: 3/%4 &H&1F-b&F2b&FQ8 @!&1!"#"$%&F-b&F2b&FQ&()*+&a8b &J&1F]b&a8 @!&1!"#"$%&F]b&a &&&&&&()*+&,- &&&&&&.*/0&H &&&&&&&&*0&1a8 &&&8b WITH Since SQL:99 Introduces another CTE Don't repeat WITH

28. CTEs are statement-scoped views: 3/%4 &H&1F-b&F2b&FQ8 @!&1!"#"$%&F-b&F2b&FQ&()*+&a8b &J&1F]b&a8 @!&1!"#"$%&F]b&a &&&&&&()*+&,- &&&&&&.*/0&H &&&&&&&&*0&1a8 &&&8b WITH Since SQL:99

29. CTEs are statement-scoped views: 3/%4 &H&1F-b&F2b&FQ8 @!&1!"#"$%&F-b&F2b&FQ&()*+&a8b &J&1F]b&a8 @!&1!"#"$%&F]b&a &&&&&&()*+&,- &&&&&&.*/0&H &&&&&&&&*0&1a8 &&&8b WITH Since SQL:99 May refer to previous CTEs

30. 3/%4 &H&1F-b&F2b&FQ8 @!&1!"#"$%&F-b&F2b&FQ&()*+&a8b &J&1F]b&a8 @!&1!"#"$%&F]b&a &&&&&&()*+&,- &&&&&&.*/0&H &&&&&&&&*0&1a8 &&&8b &F&1a8 @!&1!"#"$%&a&()*+&a8 !"#"$%&a &&()*+&J&.*/0&F&*0&1a8 WITH Since SQL:99 Third CTE

31. 3/%4 &H&1F-b&F2b&FQ8 @!&1!"#"$%&F-b&F2b&FQ&()*+&a8b &J&1F]b&a8 @!&1!"#"$%&F]b&a &&&&&&()*+&,- &&&&&&.*/0&H &&&&&&&&*0&1a8 &&&8b &F&1a8 @!&1!"#"$%&a&()*+&a8 !"#"$%&a &&()*+&J&.*/0&F&*0&1a8 WITH Since SQL:99 No comma!

32. 3/%4 &H&1F-b&F2b&FQ8 @!&1!"#"$%&F-b&F2b&FQ&()*+&a8b &J&1F]b&a8 @!&1!"#"$%&F]b&a &&&&&&()*+&,- &&&&&&.*/0&H &&&&&&&&*0&1a8 &&&8b &F&1a8 @!&1!"#"$%&a&()*+&a8 !"#"$%&a &&()*+&J&.*/0&F&*0&1a8 WITH Since SQL:99 Main query

33. CTEs are statement-scoped views: 3/%4 &H&1F-b&F2b&FQ8 @!&1!"#"$%&F-b&F2b&FQ&()*+&a8b &J&1F]b&a8 @!&1!"#"$%&F]b&a &&&&&&()*+&,- &&&&&&.*/0&H &&&&&&&&*0&1a8 &&&8b &F&1a8 @!&1!"#"$%&a&()*+&a8 !"#"$%&a &&()*+&J&.*/0&F&*0&1a8 WITH Since SQL:99 Read top down

34. WITH in an Nutshell 3/%4 are the "private methods" of SQL 3/%4 views can be referred to multiple times 3/%4 allows chaining instead of nesting 3/%4 is allowed where !"#"$% is allowed /0!")%&/0%*&,J; 3/%4&555&!"#"$%&555

35. $%"&!FH:&C:&F,< &1AC?G7RQSO8 &(9;,<AW&,CD9F&7&- &$%"&F,< &TU&!<&!FH:&C:&:<?G &&&&1AC?G7-OOOOOO-8 WITH PostgreSQL Particularities In PostgreSQL 3/%4 views are more like materialized views: 3/%4&F,<&@! 1!"#"$%&' &&&()*+&:<?G8 !"#"$%&'& &&()*+&F,< &34")"&,CD9F7-

36. $%"&!FH:&C:&F,< &1AC?G7RQSO8 &(9;,<AW&,CD9F&7&- &$%"&F,< &TU&!<&!FH:&C:&:<?G &&&&1AC?G7-OOOOOO-8 WITH PostgreSQL Particularities In PostgreSQL 3/%4 views are more like materialized views: 3/%4&F,<&@! 1!"#"$%&' &&&()*+&:<?G8 !"#"$%&'& &&()*+&F,< &34")"&,CD9F7-

37. $%"&!FH:&C:&F,< &1AC?G7RQSO8 &(9;,<AW&,CD9F&7&- &$%"&F,< &TU&!<&!FH:&C:&:<?G &&&&1AC?G7-OOOOOO-8 WITH PostgreSQL Particularities In PostgreSQL 3/%4 views are more like materialized views: 3/%4&F,<&@! 1!"#"$%&' &&&()*+&:<?G8 !"#"$%&'& &&()*+&F,< &34")"&,CD9F7- CTE doesn't know about the outer filter

38. Normal views and inline-views support "predicate pushdown": !"#"$%&' &&()*+&1 &&!"#"$%&' &&&&()*+&:<?G &&8&: 34")"&,CD9F7-c M9,PHD&4<HD&!FH: C:&:<?G&1AC?G7RQSO8 TUM9,PHD&/:E<6&!FH: &&C:&9E6&1AC?G7RQSO8 &&$C:EW&,CD9F7- WITH PostgreSQL Particularities

39. PostgreSQL 9.1+ allows /0!")%, d_L@%" and L"#"%" within 3/%4: 3/%4&E<;<,<E=AC?G&@!&1 &&&L"#"%"&()*+&GCBAF< &&&)"%d)0/0e&' 8 /0!")%&/0%*&E<G,9:H,9C: !"#"$%&'&()*+&E<;<,<E=AC?Gc WITH PostgreSQL Particularities

40. WITH Availability (SQL:99)

41. &'#()%$*+%,'-$ (Common Table Expressions)

42. WITH RECURSIVE Before SQL:99

43. (This page is intentionally left blank) WITH RECURSIVE Before SQL:99

44. Recursive common table expressions may refer to themselves in the second leg of a d0/*0&f@##g: 3/%4&)"$d)!/h"&F,<&1:8 &&@!&1!"#"$%&- &&&&&&&d0/*0&@## &&&&&&!"#"$%&:i- &&&&&&&&()*+&F,< &&&&&&&34")"&:&j&Q8 !"#"$%&'&()*+&F,<c WITH RECURSIVE Since SQL:99 Keyword

45. Recursive common table expressions may refer to themselves in the second leg of a d0/*0&f@##g: 3/%4&)"$d)!/h"&F,<&1:8 &&@!&1!"#"$%&- &&&&&&&d0/*0&@## &&&&&&!"#"$%&:i- &&&&&&&&()*+&F,< &&&&&&&34")"&:&j&Q8 !"#"$%&'&()*+&F,<c WITH RECURSIVE Since SQL:99 Column list mandatory here

46. Recursive common table expressions may refer to themselves in the second leg of a d0/*0&f@##g: 3/%4&)"$d)!/h"&F,<&1:8 &&@!&1!"#"$%&- &&&&&&&d0/*0&@## &&&&&&!"#"$%&:i- &&&&&&&&()*+&F,< &&&&&&&34")"&:&j&Q8 !"#"$%&'&()*+&F,<c WITH RECURSIVE Since SQL:99 Executed first

47. Recursive common table expressions may refer to themselves in the second leg of a d0/*0&f@##g: 3/%4&)"$d)!/h"&F,<&1:8 &&@!&1!"#"$%&- &&&&&&&d0/*0&@## &&&&&&!"#"$%&:i- &&&&&&&&()*+&F,< &&&&&&&34")"&:&j&Q8 !"#"$%&'&()*+&F,<c WITH RECURSIVE Since SQL:99 Result sent there

48. Recursive common table expressions may refer to themselves in the second leg of a d0/*0&f@##g: 3/%4&)"$d)!/h"&F,<&1:8 &&@!&1!"#"$%&- &&&&&&&d0/*0&@## &&&&&&!"#"$%&:i- &&&&&&&&()*+&F,< &&&&&&&34")"&:&j&Q8 !"#"$%&'&()*+&F,<c WITH RECURSIVE Since SQL:99 Result visible twice

49. Recursive common table expressions may refer to themselves in the second leg of a d0/*0&f@##g: 3/%4&)"$d)!/h"&F,<&1:8 &&@!&1!"#"$%&- &&&&&&&d0/*0&@## &&&&&&!"#"$%&:i- &&&&&&&&()*+&F,< &&&&&&&34")"&:&j&Q8 !"#"$%&'&()*+&F,<c WITH RECURSIVE Since SQL:99 &:& TTT &- &2 &Q 1Q&AC?G8 Once it becomes part of the final result

50. Recursive common table expressions may refer to themselves in the second leg of a d0/*0&f@##g: 3/%4&)"$d)!/h"&F,<&1:8 &&@!&1!"#"$%&- &&&&&&&d0/*0&@## &&&&&&!"#"$%&:i- &&&&&&&&()*+&F,< &&&&&&&34")"&:&j&Q8 !"#"$%&'&()*+&F,<c WITH RECURSIVE Since SQL:99 &:& TTT &- &2 &Q 1Q&AC?G8 Second leg of UNION is executed

51. Recursive common table expressions may refer to themselves in the second leg of a d0/*0&f@##g: 3/%4&)"$d)!/h"&F,<&1:8 &&@!&1!"#"$%&- &&&&&&&d0/*0&@## &&&&&&!"#"$%&:i- &&&&&&&&()*+&F,< &&&&&&&34")"&:&j&Q8 !"#"$%&'&()*+&F,<c WITH RECURSIVE Since SQL:99 &:& TTT &- &2 &Q 1Q&AC?G8 Result sent there again

52. Recursive common table expressions may refer to themselves in the second leg of a d0/*0&f@##g: 3/%4&)"$d)!/h"&F,<&1:8 &&@!&1!"#"$%&- &&&&&&&d0/*0&@## &&&&&&!"#"$%&:i- &&&&&&&&()*+&F,< &&&&&&&34")"&:&j&Q8 !"#"$%&'&()*+&F,<c WITH RECURSIVE Since SQL:99 &:& TTT &- &2 &Q 1Q&AC?G8

53. Recursive common table expressions may refer to themselves in the second leg of a d0/*0&f@##g: 3/%4&)"$d)!/h"&F,<&1:8 &&@!&1!"#"$%&- &&&&&&&d0/*0&@## &&&&&&!"#"$%&:i- &&&&&&&&()*+&F,< &&&&&&&34")"&:&j&Q8 !"#"$%&'&()*+&F,<c WITH RECURSIVE Since SQL:99 &:& TTT &- &2 &Q 1Q&AC?G8 It's a loop!

54. Recursive common table expressions may refer to themselves in the second leg of a d0/*0&f@##g: 3/%4&)"$d)!/h"&F,<&1:8 &&@!&1!"#"$%&- &&&&&&&d0/*0&@## &&&&&&!"#"$%&:i- &&&&&&&&()*+&F,< &&&&&&&34")"&:&j&Q8 !"#"$%&'&()*+&F,<c WITH RECURSIVE Since SQL:99 &:& TTT &- &2 &Q 1Q&AC?G8 n=3 doesn't match

55. Recursive common table expressions may refer to themselves in the second leg of a d0/*0&f@##g: 3/%4&)"$d)!/h"&F,<&1:8 &&@!&1!"#"$%&- &&&&&&&d0/*0&@## &&&&&&!"#"$%&:i- &&&&&&&&()*+&F,< &&&&&&&34")"&:&j&Q8 !"#"$%&'&()*+&F,<c WITH RECURSIVE Since SQL:99 &:& TTT &- &2 &Q 1Q&AC?G8 n=3 doesn't match Loop terminates

56. WITH RECURSIVE Use Cases • Row generators (previous example) (I<:<AH,<=G<A9<G18 is proprietary) • Processing graphs (don't forget the cycle detection!) • Generally said: Loops that... ‣ ... pass data to the next iteration ‣ ... need a "dynamic" abort condition

57. WITH RECURSIVE in a Nutshell 3/%4&)"$d)!/h" is the ?V9;< of SQL 3/%4&)"$d)!/h" "supports" inﬁnite loops (not in SQL Server where +@k)"$d)!/*0 is limited to 32767) Except PostgreSQL, databases generally don't require the )"$d)!/h" keyword

58. WITH RECURSIVE Availability

59. SQL:2003

60. !"#$%&

61. Pivot table: Years on the Y asis, Month on X axis: !"#"$%&'"()*& !"#$%&!'()*'+(#,+-*(.(/ (((((((((-*'+(01230('4!'(5('+67(8&+* !+,-$(!"&./"0&,10%/&2&3 &&&&&&&&&%/"0&45674&"#!"&8&"09:&;"<*= &&;)1,&4567>?5@5 &A)1+B&<'&'"() FILTER Before SQL:2003

62. SQL:2003 has !"#$%&: '%#%($)*%+&, !"#$%&'(%)*+,-./0*$12/0/*#34.2*5*6)*7849 '-./012304)!"#$%&)/56%&%).78$6)9):4)!%;, < ))!&7.)0123=>1?1 )@&7-A);*)*%+&B FILTER Since SQL:2003

63. FILTER Availability (SQL:2003)

64. .-$% and /"%#'#'.0)12

65. Show percentage of department salary: 3/%4&,C,H;=GH;HA`=J`=E<DHA,P<:, &&@!&1!"#"$%&E<Db&!d+1GH;HA`8&,C,H; &&&&&&&&()*+&<PD &&&&&&&e)*d_&MN&E<D8 !"#"$%&E<Db&<PD=9Eb&GH;HA`b &&&&&&&GH;HA`l,G5,C,H;'-OO&mn&Co&E<Dm &&()*+&<PD &&.*/0&,C,H;=GH;HA`=J`=E<DHA,P<:,&,G &&&&*0&1<PD5E<D&7&,G5E<D8 &34")"&<PD5E<D&7&K OVER Before SQL:2003

68. Show percentage of department salary: 3/%4&,C,H;=GH;HA`=J`=E<DHA,P<:, &&@!&1!"#"$%&E<Db&!d+1GH;HA`8&,C,H; &&&&&&&&()*+&<PD &&&&&&&e)*d_&MN&E<D8 !"#"$%&E<Db&<PD=9Eb&GH;HA`b &&&&&&&GH;HA`l,G5,C,H;'-OO&mn&Co&E<Dm &&()*+&<PD &&.*/0&,C,H;=GH;HA`=J`=E<DHA,P<:,&,G &&&&*0&1<PD5E<D&7&,G5E<D8 &34")"&<PD5E<D&7&K OVER Before SQL:2003 WITH intermezzo

70. e)*d_&MN&= L/!%/0$% + Aggregates OVER Before SQL:2003

71. Build aggregates without e)*d_&MN: !"#"$%&E<Db&<PD=9Eb&GH;HA`b &&&&&&&GH;HA`l1A4B>.C.-DE &&&&&&&&&&&&&&'F$%BG"%#3#3'H&)*&97+E &&&&&&&&&&&&&'&-OO&mn&Co&E<Dm &&()*+&<PD OVER Since SQL:2003

72. OVER How It Works E<D GH;HA` - -OOO ROOO 22 -OOO ROOO 22 -OOO ROOO QQQ -OOO ROOO QQQ -OOO ROOO QQQ -OOO ROOO !"#"$%&E<Db& &&&&&&&GH;HA`b &&&&&&&!d+1GH;HA`8 &&&&&&&*h")&18 &&()*+&<PDc

75. %&E<Db& &&GH;HA`b &&!d+1GH;HA`8 &&*h")1_@)%/%/*0&MN&E<D8 +&<PDc E<D GH;HA` ,G - -OOO -OOO 22 -OOO 2OOO 22 -OOO 2OOO QQQ -OOO QOOO QQQ -OOO QOOO QQQ -OOO QOOO OVER How It Works

78. OVER in a Nutshell *h") may follow any aggregate function *h") deﬁnes which rows are visible at each row (it does not limit the result in any way) *h")18 makes all rows visible at every row *h")1_@)%/%/*0&MN x) segregates like e)*d_&MN

79. .-$% and .%3$%)12

80. OVER Before SQL:2003 Calculating a running total: !"#"$%&,69Eb&>H;B<b &&&&&&&1!"#"$%&!d+1>H;B<8 &&&&&&&&&&()*+&,AH:GHF,9C:G&,62 &&&&&&&&&34")"&HF:,&7&K &&&&&&&&&&&@0L&,625,69E&j7&,6-5,69E8&JH; &&()*+&,AH:GHF,9C:G&,6- &34")"&HF:,&7&K &*)L")&MN&,69E

85. OVER Before SQL:2003 Before SQL:2003 running totals were awkward: ‣ Requires a scalar sub-select or self-join ‣ Poor maintainability (reparative clauses) ‣ Poor performance The only real answer was: Do it in the application

86. With SQL:2003 you can narrow the window: !"#"$%&,69Eb&>H;B<b &&&&&&&!d+1>H;B<8 &&&&&&&*h")1'%($%&)*&8IJ9 &&&&&&&&&&&&%'K1 &&&&&&&&&&&&)$#K$$H&AH)'AH($(&G%$2$(3HL &&&&&&&&&&&&&&&&"H(&2A%%$H#&%'K8&JH; &&()*+&,AH:GHF,9C:G&,6- &34")"&HF:,&7&K &*)L")&MN&,69E OVER Since SQL:2003

87. With *h")&1*)L")&MN&a8 a new type of functions makes sense: ‣ )*3=0d+M") ‣ Ranking functions:& )@0pb&L"0!"=)@0pb&_")$"0%=)@0pb $d+"=L/!% OVER Since SQL:2003

88. OVER Availability (SQL:2003)

89. !"#$"%&'()*+

90. WITHIN GROUP Before SQL:2003 Getting the median: !"#"$%&'()*+, &&-./0&'+1+&'( &&2/34&'+1+&'5 &&&&/4&6'()*+,&7&'5)*+, &&&&&&&/.&6'()*+,8'5)*+,&94:&'();'7'5);'<< &=./>?&@A&'()*+, B9C34=&DEFG16H<&8& &&&&&&&6!"#"$%&-#//.6$/>4%6H<I5< &&&&&&&&&&-./0&'+1+<

91. WITHIN GROUP Since SQL:2003 SQL:2003 introduced ordered-set functions... !"#"$%&'"($")%*#"+,*!$-./01 &&&&&&&2*%3*)&4(56'&-5(,"(&78&9:;1 &&<(5=&>:?: ...and hypothetical-set functions to say which rank a hypothetical row would have: &!"#"$%&(@)A-BCD1 &&&&&&&&2*%3*)&4(56'&-5(,"(&78&9:;1 &&&<(5=&>:?: Median Which value?

92. WITHIN GROUP Availability

93. SQL:2008

94. .-$%

95. Calculate the difference to a previous row: 3/%4&:BPJ<A<E=EH,H&@!&1 &!"#"$%&'b &&&&&&&&)*3=0d+M")18&*h")1*)L")&MN&68&A: &&&()*+&EH,H8& !"#"$%&FBA5'b&FBA5JH;H:F<TDA<>5JH;H:F< &&()*+&&&&&&:BPJ<A<E=EH,H&FBA &&#"(%&.*/0&:BPJ<A<E=EH,H&DA<> &&&&*0&1FBA5A:&7&DA<>5A:T-8 OVER Before SQL:2008

96. Calculate the difference to a previous row: 3/%4&:BPJ<A<E=EH,H&@!&1 &!"#"$%&'b &&&&&&&&)*3=0d+M")18&*h")1*)L")&MN&68&A: &&&()*+&EH,H8& !"#"$%&FBA5'b&FBA5JH;H:F<TDA<>5JH;H:F< &&()*+&&&&&&:BPJ<A<E=EH,H&FBA &&#"(%&.*/0&:BPJ<A<E=EH,H&DA<> &&&&*0&1FBA5A:&7&DA<>5A:T-8 OVER Before SQL:2008

97. SQL:2008 can access other rows directly: !"#"$%&'b&JH;H:F<&T&#@e1JH;H:F<8 &&&&&&&&&&&&&&&&&&&&*h")1*)L")&MN&68 &&()*+&EH,H Available functions: &#"@L&l&#@e &(/)!%=h@#d"&l&#@!%=h@#d" &0%4=h@#d"1FC;b&:8&()*+&(/)!%l#@!% &&&&&&&&&&&&&&&&&&&)"!_"$%l/e0*)"&0d##! OVER Since SQL:2008 Not supported by PostgreSQL (as of 9.4)

98. OVER Availability (SQL:2008)

99. 4$#*()4'%,#

100. Limit the number of selected rows: !"#"$%&' &&()*+&1!"#"$%&'b &&&&&&&&)*3=0d+M")18&*h")1*)L")&MN&68&A: &&&()*+&EH,H8&:BPJ<A<E=EH,H &34")"&A:&j7-O FETCH FIRST Before SQL:2008

101. Limit the number of selected rows: !"#"$%&' &&()*+&1!"#"$%&'b &&&&&&&&)*3=0d+M")18&*h")1*)L")&MN&68&A: &&&()*+&EH,H8&:BPJ<A<E=EH,H &34")"&A:&j7-O FETCH FIRST Before SQL:2008 PostgreSQL does not optimize this properly!

102. Limit the number of selected rows: !"#"$%&' &&()*+&1!"#"$%&'b &&&&&&&&)*3=0d+M")18&*h")1*)L")&MN&68&A: &&&()*+&EH,H8&:BPJ<A<E=EH,H &34")"&A:&j7-O FETCH FIRST Before SQL:2008 Dammit! Let's take LIMIT (or TOP)

103. SQL:2008 has ("%$4&(/)!%&:&)*3!&*0#N: !"#"$%&' &&()*+&EH,H &*)L")&MN&6 &("%$4&(/)!%&-O&)*3!&*0#N FETCH FIRST Since SQL:2008

104. FETCH FIRST Availability

105. SQL:2011

106. .44,$#

107. Skip 10 rows, then deliver only the next 10: !"#"$%&' &&()*+&1!"#"$%&'b &&&&&&&&)*3=0d+M")18&*h")1*)L")&MN&68&A: &&&()*+&EH,H &&("%$4&(/)!%&2O&)*3!&*0#N 8&:BPJ<A<E=EH,H &34")"&A:&U&-O OFFSET Before SQL:2011

108. SQL:2011 introduced *((!"%, unfortunately: !"#"$%&' &&()*+&EH,H &*)L")&MN&6 'MM1$#&:;&%'K1 ("%$4&0"k%&-O&)*3!&*0#N OFFSET Since SQL:2011

109. OFFSET is EVIL http://use-the-index-luke.com/no-offset

110. OFFSET Availability (SQL:2011)

111. &'#(.+#).-$%!"/,

112. Prior SQL:2011 it was not possible to deﬁne constraints that avoid overlapping periods. Workarounds are possible, but no fun: $)"@%"&%)/ee") WITHOUT OVERLAPS Before SQL:2011 id begin end 1 8:00 9:00 1 9:00 11:00 1 10:00 12:00

113. SQL:2011 introduced temporal and bi-temporal features —e.g., for constraints: _)/+@)N&p"N&19Eb&+7-J<9&K3#N'A#&'F$%!"G18 PostgreSQL 9.2 introduced range types and "exclusive constraints" which can accomplish the same effect: "k$#dL"&d!/0e&I9G, &&&&&&&&19E&3/%4&7b&+7-J<9&K3#N&OO8 WITHOUT OVERLAPS Since SQL:2011

114. SQL:2011 goes far beyond 3/%4*d%&*h")#@_!. Please read these papers to get the idea: Temporal features in SQL:2011 http://cs.ulb.ac.be/public/_media/teaching/infoh415/tempfeaturessql2011.pdf What's new in SQL:2011? http://www.sigmod.org/publications/sigmod-record/1203/pdfs/10.industry.zemke.pdf Temporal/Bi-Temporal SQL

115. WITHOUT OVERLAPS Availability

116. About @MarkusWinand Tuning developers for high SQL performance Training & tuning: http://winand.at/ Author of: http://sql-performance-explained.com/ Geeky blog: http://use-the-index-luke.com

Modern SQL in PostgreSQL

Markus Winand

Transcript

Introducing the official SlideShare app

Stunning, full-screen experience for iPhone and Android

Text the download link to your phone

Modern SQL in PostgreSQL

Markus Winand

Transcript