MySQL Query tuning 101

1. MySQL Query Tuning 101 Sveta Smirnova, Alexander Rubin April, 16, 2015

2. • Introduction: where to ﬁnd slow queries • Indexes: why and how do they work • All about EXPLAIN • More tools • Where to ﬁnd more information Agenda 2

3. • You develop an application and find out that some queries are running slow • After a while you find some slow queries in the slow query log • All such queries are always slow • We would not talk about cases when concurrency affects performance When you see slow query first 3

4. • MySQL has to do some job to execute a select query • In the worst case scenario it will do a full table scan • CREATE INDEX • Incorrect index can be used by MySQL Why query can run slow 4

5. select * from table select * from table where id=12 1 2 5 6 7 9 12 16 18 21 22 23 24 25 Full table scan 5

6. • When you add index (except for MEMORY) MySQL will use B-Tree • Support equality and “range” operations MySQL Indexes 6

7. • select * from table where id = 12 • Scan thru the tree and go directly to 1 leaf • Stop B-Tree: Equality search 7

8. • select * from table where id in (6, 12, 18) • Scan thru the tree and visit many leafs/nodes B-Tree: Range 8

9. • EXPLAIN – Estimates what happens during query execution – EXTENDED – FORMAT=JSON – PARTITIONS • INFORMATION SCHEMA.OPTIMIZER TRACE – Real data, collected after query was executed – Advanced topic How to ﬁnd out how MySQL uses indexes 9

10. mysql> explain select * from t1G *************************** 1. row *************************** ... rows: 12 Extra: NULL mysql> explain select * from t1 where f2=12G *************************** 1. row *************************** ... key: NULL ... rows: 12 Extra: Using where Same number of examined rows for both queries Effect of indexes: before 10

11. mysql> alter table t1 add index(f2); Query OK, 12 rows affected (0.07 sec) Records: 12 Duplicates: 0 Warnings: 0 mysql> explain select * from t1 where f2=12G *************************** 1. row *************************** ... key: f2 key_len: 5 ref: const rows: 1 Extra: NULL 1 row in set (0.00 sec) Much more eﬀective! Only 1 row examined Effect of indexes: after 11

12. mysql> explain extended select * from t1 join t2 where t1.int_key=1; +----+-------------+-------+-------+---------------+---------+---------+-------+------+----------+------------------+ | id | select_type | table | type | possible_keys | key | key_len | ref | rows | filtered | Extra | +----+-------------+-------+-------+---------------+---------+---------+-------+------+----------+------------------+ | 1 | SIMPLE | t1 | ref | int_key,ik | int_key | 5 | const | 4 | 100.00 | NULL | | 1 | SIMPLE | t2 | index | NULL | pk | 9 | NULL | 6 | 100.00 | Using index; | Using join buffer | (Block Nested Loop) | +----+-------------+-------+-------+---------------+---------+---------+-------+------+----------+------------------+ 2 rows in set, 1 warning (0.00 sec) Note (Code 1003): /* select#1 */ select ‘test‘.‘t1‘.‘pk‘ AS ‘pk‘,‘test‘.‘t1‘.‘int_key‘ AS ‘int_key‘,‘test‘.‘t2‘.‘pk‘ AS ‘pk‘,‘test‘.‘t2‘.‘int_key‘ AS ‘int_key‘ from ‘test‘.‘t1‘ join ‘test‘.‘t2‘ where (‘test‘.‘t1‘.‘int_key‘ = 1) Number of select Select type Tables, for which information is printed How data is accessed Possible keys Key, which was actually used Length of the key Which columns were compared with the index Number of examined rows % of ﬁltered rows rows x ﬁltered / 100 — number of rows, which will be joined with another table Additional information Table, for which information is printed Product of rows here: how many rows in all tables will be accessed For this example estimated value is 4*6 = 24 Actual (optimized) query as executed by MySQL Server EXPLAIN: overview 12

13. mysql> explain extended select * from t1 join t2 where... +----+-------------+-------+-------+*** | id | select_type | table | type |*** +----+-------------+-------+-------+*** | 1 | SIMPLE | t1 | ref |*** | 1 | SIMPLE | t2 | index |*** +----+-------------+-------+-------+*** 2 rows in set, 1 warning (0.00 sec) SIMPLE;PRIMARY;UNION;DEPENDENT UNION;UNION RESULT; SUBQUERY;DEPENDENT SUBQUERY;DERIVED;MATERIALIZED system const eq ref ref fulltext ref or null index merge unique subquery index subquery range index ALL EXPLAIN in details 13

14. mysql> explain extended select * from t1 join t2 where t1.int_key=1; ***+---------------+---------+---------+-------+*** ***| possible_keys | key | key_len | ref |*** ***+---------------+---------+---------+-------+*** ***| int_key,ik | int_key | 5 | const |*** ***| NULL | pk | 9 | NULL |*** ***+---------------+---------+---------+-------+*** 2 rows in set, 1 warning (0.00 sec) Keys, which can be used for resolving the query Only one key was actually used Actual length of the key (Important for multiple-column keys) Which columns were compared with the index Constant Numeric in our case Index used to resolve rows EXPLAIN in details: keys 14

15. mysql> explain extended select * from t1 join t2 where t1.int_key=1; ***+-------+----------+----------------------------------------------------+ ***| rows | filtered | Extra | ***+-------+----------+----------------------------------------------------+ ***| 4 | 100.00 | NULL | ***| 6 | 100.00 | Using index; Using join buffer (Block Nested Loop) | ***+-------+----------+----------------------------------------------------+ 2 rows in set, 1 warning (0.00 sec) Number of rows accessed % of rows ﬁltered Additional information: how query is resolved Using ﬁlesort Using temporary etc. 4X6 = 24 All rows used EXPLAIN in details: rows 15

16. • MySQL or Percona Server 5.6 • Employees test database – XML: Fusheng Wang and Carlo Zaniolo – SQL: Giuseppe Maxia and Patrick Crews – More information • Download: https://launchpad.net/test-db/ • Install: – cd employees db – mysql <employees.sql EXPLAIN type by example: setup 16

17. mysql> explain select count(*) from employees where hire_date > ’1995-01-01’ ********************** 1. row ********************** id: 1 select_type: SIMPLE table: employees type: ALL possible_keys: NULL key: NULL key_len: NULL ref: NULL rows: 300157 Extra: Using where 1 row in set (0.00 sec) All rows in the table examined Worst plan ever! EXPLAIN type by example: ALL 17

18. mysql> explain select count(*) from titles where title=’Senior Engineer’G ********************** 1. row ********************** id: 1 select_type: SIMPLE table: titles type: index possible_keys: NULL key: emp_no key_len: 4 ref: NULL rows: 444033 Extra: Using where; Using index 1 row in set (0.11 sec) No row in the table was accessed to resolve the query! Only index used Still all records in the index were scanned EXPLAIN type by example: index 18

19. • We need to add index to table employees ﬁrst • mysql> alter table employees -> add index(hire_date); Query OK, 0 rows affected (3.48 sec) Records: 0 Duplicates: 0 Warnings: 0 EXPLAIN type by example: range 19

20. mysql> explain select count(*) from employees where hire_date > ’1995-01-01’ ********************** 1. row ********************** id: 1 select_type: SIMPLE table: employees type: range possible_keys: hire_date key: hire_date key_len: 3 ref: NULL rows: 68654 Extra: Using where; Using index 1 row in set (0.00 sec) Only rows from given range used Compare with ALL: 300157/68654 = 4.3720 4 times less rows examined! EXPLAIN type by example: range 20

21. • Consists of two or more columns • Only leftmost part used • mysql> alter table City add key comb(CountryCode, District, Population), drop key CountryCode; Combined indexes 21

22. mysql> explain select * from City where CountryCode = ’USA’G ********************** 1. row ****************** table: City type: ref possible_keys: comb key: comb key_len: 3 ref: const rows: 273 Uses ﬁrst ﬁeld from the comb key Combined indexes: example 1 22

23. mysql> explain select * from City where -> District = ’California’ and population > 10000G ********************** 1. row ****************** table: City type: ALL possible_keys: NULL key: NULL key_len: NULL ref: NULL rows: 3868 Can’t use combined index: not a leftmost part Does not have the CountryCode in the where clause = can’t use comb index Combined indexes: example 2 23

24. • Key len = total size (in bytes) of index parts used • Index: comb(CountryCode, District, Population) • Explain: Fields: key: comb CountryCode char(3) key len: 3 District char(20) Population int(11) 3 ->Char(3) ->First ﬁeld is used Combined indexes: key len 24

25. • Covered index = cover all ﬁelds in query • select name from City where CountryCode = ’USA’ and District = ’Alaska’ and population > 10000 mysql> alter table City add key cov1(CountryCode, District, population, name); Uses all ﬁelds in the query in particular order 1. Where part 2. Group By/Order (not used now) 3. Select part Covered indexes 25

26. mysql> explain select name from City where CountryCode = ’USA’ -> and District = ’Alaska’ and population > 10000G *************************** 1. row *********** table: City type: range possible_keys: cov1 key: cov1 key_len: 27 ref: NULL rows: 1 Extra: Using where; Using index Covered index is used MySQL will only use index Will not go to the data ﬁle EXPLAIN by example: covered indexes 26

27. mysql> explain select * from dept_emp where dept_no = ’d005’G ************************ 1. row ************************ id: 1 select_type: SIMPLE table: dept_emp type: ref possible_keys: dept_no key: dept_no key_len: 4 ref: const rows: 145708 Extra: Using where 1 row in set (0.00 sec) EXPLAIN type by example: ref 27

28. mysql> explain select * from dept_manager join employees using(emp_no) limit 10G ************************ 1. row ************************ id: 1 select_type: SIMPLE table: dept_manager type: ALL possible_keys: PRIMARY,emp_no key: NULL key_len: NULL ref: NULL rows: 24 Extra: EXPLAIN type by example: eq ref 28

29. ************************ 2. row ************************ id: 1 select_type: SIMPLE table: employees type: eq_ref possible_keys: PRIMARY key: PRIMARY key_len: 4 ref: employees.dept_manager.emp_no rows: 1 Extra: 2 rows in set (0.00 sec) EXPLAIN type by example: eq ref 29

30. mysql> explain select * from departments where dept_no=’d005’G ************************ 1. row ************************ id: 1 select_type: SIMPLE table: departments type: const possible_keys: PRIMARY key: PRIMARY key_len: 4 ref: const rows: 1 Extra: 1 row in set (0.00 sec) EXPLAIN type by example: const 30

31. • Limitations – It shows estimates only • Extensions – EXTENDED – PARTITIONS – FORMAT=JSON EXPLAIN limitations and extensions 31

32. mysql> explain partitions select count(*) -> from employees_part where hire_date > ’1991-01-01’G ************************ 1. row ************************ id: 1 select_type: SIMPLE table: employees_part partitions: p1,p2 type: index possible_keys: NULL key: PRIMARY key_len: 7 ref: NULL rows: 135214 Extra: Using where; Using index EXPLAIN PARTITIONS 32

33. • Gives more information – Real execution path of the query – Pushed conditions – Temporary table and index creation are more precise – Reﬂects execution order of ”group by” and ”order by” operations – Displays table materializations EXPLAIN FORMAT=JSON 33

34. mysql> explain format=json insert into salaries(emp_no, from_date) select emp_no, min(from_date) from titles group by emp_noG *********************** 1. row *********************** EXPLAIN: { "query_block": { "select_id": 1, "grouping_operation": { "using_filesort": false, "table": { "table_name": "titles", EXPLAIN FORMAT=JSON 34

35. mysql> explain insert into salaries(emp_no, from_date) -> select emp_no, min(from_date) from titles group by emp_noG ********************** 1. row ********************** id: 1 select_type: SIMPLE table: titles type: index possible_keys: NULL key: emp_no EXPLAIN for DML: not only SELECT 35

36. mysql> explain insert into salaries(emp_no, from_date) select emp_no, min(from_date) from titles group by emp_noG ... key_len: 4 ref: NULL rows: 444058 Extra: Using index 1 row in set, 2 warnings (0.00 sec) Warning (Code 1364): Field ’salary’ doesn’t have a default value Warning (Code 1364): Field ’to_date’ doesn’t have a default value EXPLAIN for DML: not only SELECT 36

37. • Status variables ’Handler %’ • Performance schema – events stages % – events statements % • INFORMATION SCHEMA.OPTIMIZER TRACE Other tools 37

38. • MySQL User Reference Manual • MySQL Troubleshooting book • High Performance MySQL book • MySQL Performance Blog • Planet MySQL • Troubleshooting Performance Companion slides More to learn 38

39. Thank you! 39

40. ? Questions? 40