Highly Available MySQL/PHP Applications with mysqlnd

1. Highly Available PHP/MySQL Applications with mysqlnd Jervin Real PHP-UGPH, June 2014

2. Highly Available Applications • … how much your application is able to serve its end users within a time period

3. Highly Available Applications • … how much your application is able to serve its end users within a time period • Usually referred to as 99[..]% uptime

4. Highly Available Applications • … how much your application is able to serve its end users within a time period • Usually referred to as 99[..]% uptime • It actually depends on whose perspective

5. Highly Available Applications Keep PHP<>MySQL downtime to a minimum

6. WTF are you saying? I’m a “devloper"! No no no no no no!

7. Lack of knowledge

8. Limited resources

9. How?

10. MySQL

11. MySQL HA • MySQL Native Replication (Single Primary)

12. MySQL HA • MySQL Native Replication (Single Primary) • MySQL Native Replication (Multi-Master)

13. MySQL HA • MySQL Native Replication (Single Primary) • MySQL Native Replication (Multi-Master) • MySQL NDB Cluster

14. MySQL HA • MySQL Native Replication (Single Primary) • MySQL Native Replication (Multi-Master) • MySQL NDB Cluster • Galera based

15. MySQL HA • MySQL Native Replication (Single Primary) • MySQL Native Replication (Multi-Master) • MySQL NDB Cluster • Galera based • Percona XtraDB Cluster • MariaDB Galera Cluster • MySQL+Galera

16. MySQL HA • MySQL Native Replication (Single Primary) • MySQL Native Replication (Multi-Master) • MySQL NDB Cluster • Galera based • Percona XtraDB Cluster • MariaDB Galera Cluster • MySQL+Galera • Tungsten Replicator

17. MySQL HA • MySQL Native Replication (Single Primary) • MySQL Native Replication (Multi-Master) • MySQL NDB Cluster • Galera based • Percona XtraDB Cluster • MariaDB Galera Cluster • MySQL+Galera • Tungsten Replicator • … and many more

18. • MySQL Master HA MySQL HA (External Support)

19. MySQL HA (External Support) • MySQL Master HA • Percona Replication Manager (Pacemaker)

20. MySQL HA (External Support) • MySQL Master HA • Percona Replication Manager (Pacemaker) • Haproxy

21. MySQL HA (External Support) • MySQL Master HA • Percona Replication Manager (Pacemaker) • Haproxy • Load Balancers

22. MySQL HA (External Support) • MySQL Master HA • Percona Replication Manager (Pacemaker) • Haproxy • Load Balancers • MySQL Multi Master

23. MySQL HA (External Support) • MySQL Master HA • Percona Replication Manager (Pacemaker) • Haproxy • Load Balancers • MySQL Multi Master

24. PHP (Application)

25. HA via Application • Traditionally complex to implement • Unnecessary overhead • No single point of failure

26. mysqlnd

27. mysqlnd • MySQL Native Driver

28. mysqlnd • MySQL Native Driver • Written C (as all extensions should be!)

29. mysqlnd • MySQL Native Driver • Written C (as all extensions should be!) • MySQL Client Libraries no more

30. mysqlnd • MySQL Native Driver • Written C (as all extensions should be!) • MySQL Client Libraries no more • Smaller memory footprint in many cases

31. mysqlnd • MySQL Native Driver • Written C (as all extensions should be!) • MySQL Client Libraries no more • Smaller memory footprint in many cases • Performance statistics

32. mysqlnd • MySQL Native Driver • Written C (as all extensions should be!) • MySQL Client Libraries no more • Smaller memory footprint in many cases • Performance statistics • Compression

33. mysqlnd • MySQL Native Driver • Written C (as all extensions should be!) • MySQL Client Libraries no more • Smaller memory footprint in many cases • Performance statistics • Compression • Plugins

34. mysqlnd Plugins • mysqlnd_qc - query result cache

35. mysqlnd Plugins • mysqlnd_qc - query result cache • mysqlnd_mux - connection multiplexing

36. mysqlnd Plugins • mysqlnd_qc - query result cache • mysqlnd_mux - connection multiplexing • mysqlnd_uh - user handler

37. mysqlnd Plugins • mysqlnd_qc - query result cache • mysqlnd_mux - connection multiplexing • mysqlnd_uh - user handler • mysqlnd_memcache - Memcache (MySQL interface)

38. mysqlnd Plugins • mysqlnd_qc - query result cache • mysqlnd_mux - connection multiplexing • mysqlnd_uh - user handler • mysqlnd_memcache - Memcache (MySQL interface) • mysqlnd_ms - replication and load balancing

39. mysqlnd_ms

40. mysqlnd_ms Supported Clusters • All previously mentioned with few limitations

41. Limitations (Master-Slave) • Master is a single point of failure • Need external help for failover • . . or, shoot the other node in the foot, this is still HA!

42. Limitations (Master-Slave)

45. Limitations (Master-Master) • Risky, even if you try segregating writes per master you can still lose data

46. Limitations (Master-Master) db1 db1

49. Limitations (Master-Master) db1 + db2 • Not good if sync_binlog = 0 AND/OR innodb_flush_log_at_trx_commit <> 1 • Not really recommended in general, use appropriate technologies instead

50. Some More Limitations • Lazy connections is on by default • Each request will create its own connections when needed, persistent connections may be required • https://bugs.php.net/bug.php?id=67564 • https://bugs.php.net/bug.php?id=67565 • https://bugs.php.net/bug.php?id=67574 • Be careful with multi-statements and read-write splitting, they are likely to be executed on slave! SELECT * FROM tbl; DELETE FROM tbl;

51. • Single Primary - one writable node • Multiple Primary - all nodes writable mysqlnd_ms Supported Clusters

52. • Master-Slave mysqlnd_ms Supported Clusters

53. • Master-Slave • Declare 2 clusters mysqlnd_ms Supported Clusters

54. • Master-Slave • Declare 2 clusters • When primary fails, mark it permanently dead and switch to secondary mysqlnd_ms Supported Clusters

55. • Master-Slave • Declare 2 clusters • When primary fails, mark it permanently dead and switch to secondary { "primary": { "master": { "master_1": { "host": "127.0.0.1", "port": 33001 } }, "slave": { "slave_0": { "host": "127.0.0.1", "port": 33002 } } }, "standby": { "master": { "master_1": { "host": "127.0.0.1", "port": 33002 } }, "slave": { } } } mysqlnd_ms Supported Clusters

56. • Master-Slave • Declare 2 clusters • When primary fails, mark it permanently dead and switch to secondary • Works better with master-master! { "primary": { "master": { "master_1": { "host": "127.0.0.1", "port": 33001 } }, "slave": { "slave_0": { "host": "127.0.0.1", "port": 33002 } } }, "standby": { "master": { "master_1": { "host": "127.0.0.1", "port": 33002 } }, "slave": { } } } mysqlnd_ms Supported Clusters

57. • Master-Slave Demo • https://github.com/dotmanila/demo-me/phpugph201407/ • master-slave.ini • mysqlnd_ms_ms.ini • master-slave.php mysqlnd_ms Supported Clusters

58. • Multi-Masters (NDB, Galera) • Declare all nodes as masters mysqlnd_ms Supported Clusters

59. • Multi-Masters (NDB, Galera) mysqlnd_ms Supported Clusters

60. • Multi-Masters (NDB, Galera) • Declare all nodes as masters mysqlnd_ms Supported Clusters { "primary": { "master": { "master_1": { "host": "192.168.56.44", "port": "3306" }, "master_2": { "host": "192.168.56.43", "port": "3306" }, "master_3": { "host": "192.168.56.42", "port": "3306" } }, "slave": { }, "filters": { "random": [ ] }, "failover": { "strategy": "loop_before_master", "remember_failed": true } } }

61. • Multi-Masters (NDB, Galera) mysqlnd_ms Supported Clusters { "primary": { "master": { "master_1": { "host": "192.168.56.44", "port": "3306" }, "master_2": { "host": "192.168.56.43", "port": "3306" }, "master_3": { "host": "192.168.56.42", "port": "3306" } }, "slave": { }, "filters": { "random": [ ] }, "failover": { "strategy": "loop_before_master", "remember_failed": true } } }

62. • Multi-Masters (NDB, Galera) • Declare all nodes as masters • roundrobin - single node writer (first node in config) mysqlnd_ms Supported Clusters { "primary": { "master": { "master_1": { "host": "192.168.56.44", "port": "3306" }, "master_2": { "host": "192.168.56.43", "port": "3306" }, "master_3": { "host": "192.168.56.42", "port": "3306" } }, "slave": { }, "filters": { "random": [ ] }, "failover": { "strategy": "loop_before_master", "remember_failed": true } } }

63. • Multi-Masters (NDB, Galera) • Declare all nodes as masters • roundrobin - single node writer (first node in config) • random - all nodes mysqlnd_ms Supported Clusters { "primary": { "master": { "master_1": { "host": "192.168.56.44", "port": "3306" }, "master_2": { "host": "192.168.56.43", "port": "3306" }, "master_3": { "host": "192.168.56.42", "port": "3306" } }, "slave": { }, "filters": { "random": [ ] }, "failover": { "strategy": "loop_before_master", "remember_failed": true } } }

64. • Multi-Masters (NDB, Galera) • Declare all nodes as masters • roundrobin - single node writer (first node in config) • random - all nodes • remember_failed is not what it says it is { "primary": { "master": { "master_1": { "host": "192.168.56.44", "port": "3306" }, "master_2": { "host": "192.168.56.43", "port": "3306" }, "master_3": { "host": "192.168.56.42", "port": "3306" } }, "slave": { }, "filters": { "random": [ ] }, "failover": { "strategy": "loop_before_master", "remember_failed": true } } } mysqlnd_ms Supported Clusters

65. • Master-Master Demo • https://github.com/dotmanila/demo-me/phpugph201407/ • master-master.ini • mysqlnd_ms_mm.ini • master-master.php mysqlnd_ms Supported Clusters

66. Am I Using mysqlnd? (rpm) [revin@forge ~]$ rpm -‐q php-‐mysql.x86_64 -‐-‐requires […] libmysqlclient_r.so.16()(64bit) libmysqlclient_r.so.16(libmysqlclient_16)(64bit) libmysqlclient.so.16()(64bit) libmysqlclient.so.16(libmysqlclient_16)(64bit) […]

67. [revin@forge ~]$ rpm -‐q php-‐mysqlnd.x86_64 -‐-‐requires php-‐pdo(x86-‐64) = 5.4.30-‐36.el6.art rpmlib(VersionedDependencies) <= 3.0.3-‐1 rpmlib(FileDigests) <= 4.6.0-‐1 rpmlib(PayloadFilesHavePrefix) <= 4.0-‐1 rpmlib(CompressedFileNames) <= 3.0.4-‐1 libc.so.6()(64bit) libc.so.6(GLIBC_2.2.5)(64bit) libc.so.6(GLIBC_2.3.4)(64bit) libc.so.6(GLIBC_2.4)(64bit) libpthread.so.0()(64bit) libpthread.so.0(GLIBC_2.2.5)(64bit) rtld(GNU_HASH) rpmlib(PayloadIsXz) <= 5.2-‐1 Am I Using mysqlnd? (rpm)

68. [revin@forge ~]$ php -‐i […] ! mysql ! MySQL Support => enabled Active Persistent Links => 0 Active Links => 0 Client API version => 5.1.59 MYSQL_MODULE_TYPE => external MYSQL_SOCKET => /var/lib/mysql/mysql.sock MYSQL_INCLUDE => -‐I/usr/include/mysql MYSQL_LIBS => -‐L/usr/lib64/mysql -‐lmysqlclient Am I Using mysqlnd? (phpinfo)

69. [revin@forge ~]$ php -‐i […] ! mysqlnd ! mysqlnd => enabled Version => mysqlnd 5.0.10 -‐ 20111026 -‐ $Id: c85105d7c6f7d70d609bb4c000257868a40840ab $ Compression => supported SSL => supported Command buffer size => 4096 Read buffer size => 32768 Read timeout => 31536000 Collecting statistics => Yes Collecting memory statistics => No Tracing => n/a Loaded plugins => mysqlnd,example,debug_trace, auth_plugin_mysql_native_password, auth_plugin_mysql_clear_password API Extensions => mysql,mysqli,pdo_mysql Am I Using mysqlnd? (phpinfo)

70. Conclusion • Yes, we can achieve HA with mysqlnd_ms • Not for the faint of heart • Do not rely on for HA (writes) on critical production systems • Good for intended use case, rw-splitting

71. Q&A • http://dotmanila.com/blog/ • http://www.mysqlperformanceblog.com/ • @dotmanila Percona is Hiring!! http://www.percona.com/about-us/careers/open-positions

72. $t = true AND false; echo (int) $t;

Highly Available MySQL/PHP Applications with mysqlnd

Jervin Real

Transcript