New data dictionary an internal server api that matters
Upcoming SlideShare
Loading in...5
×
 

New data dictionary an internal server api that matters

on

  • 139 views

A new Data Dictionary based on transactional tables is being developed for the MySQL server. That project is a huge step forward improving many aspects of the server. The new Data Dictionary provides ...

A new Data Dictionary based on transactional tables is being developed for the MySQL server. That project is a huge step forward improving many aspects of the server. The new Data Dictionary provides API which is intended to be used by all the participants of the MySQL Server Ecosystem. The slides make a brief introduction about what general Data Dictionary is, provide overview of the MySQL traditional Data Dictionary and its limitations. Then, the presentation shows the design goals of the new Data Dictionary and sketch the main architectural decisions. It also provides the description of a few visible advantages for the MySQL users.

These are the slides for my session on OOW 2014.

Statistics

Views

Total Views
139
Views on SlideShare
139
Embed Views
0

Actions

Likes
0
Downloads
2
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • This is a Safe Harbor Front slide, one of two Safe Harbor Statement slides included in this template. <br /> <br /> One of the Safe Harbor slides must be used if your presentation covers material affected by Oracle’s Revenue Recognition Policy <br /> <br /> To learn more about this policy, e-mail: Revrec-americasiebc_us@oracle.com <br /> <br /> For internal communication, Safe Harbor Statements are not required. However, there is an applicable disclaimer (Exhibit E) that should be used, found in the Oracle Revenue Recognition Policy for Future Product Communications. Copy and paste this link into a web browser, to find out more information.   <br /> <br /> http://my.oracle.com/site/fin/gfo/GlobalProcesses/cnt452504.pdf <br /> <br /> For all external communications such as press release, roadmaps, PowerPoint presentations, Safe Harbor Statements are required. You can refer to the link mentioned above to find out additional information/disclaimers required depending on your audience.
  • This is a sample Picture with Caption Layout slide ideal for including a picture with a brief descriptive statement. <br /> <br /> To Replace the Picture on this Sample Slide (this applies to all slides in this template that contain replaceable pictures) <br /> <br /> Select the sample picture and press Delete. Click the icon inside the shape to open the Insert Picture dialog box. Navigate to the location where the picture is stored, select desired picture and click on the Insert button to fit the image proportionally within the shape. <br /> <br /> Note: Do not right-click the image to change the picture inside the picture placeholder. This will change the frame size of the picture placeholder. Instead, follow the steps outlined above. <br /> <br />

New data dictionary an internal server api that matters Presentation Transcript

  • 1. New Data Dictionary: An Internal Server API That Matters Alexander Nozdrin, Principle Software Developer Copyright Copyright © © 2014, 2014, Oracle Oracle and/and/or or its its affiliates. affiliates. All All rights rights reserved. reserved. |
  • 2. Safe Harbor Statement The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | 2
  • 3. MySQL Community Reception @ Oracle OpenWorld Mingle with the MySQL community and the MySQL team from Oracle for a fun and informative evening! • Time: September 30 (Tue) @ 7pm • Jillian’s at Metreon 175 Fourth Street, San Francisco, CA At the corner of Howard and 4th st.; only 2-min walk from Moscone Center Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
  • 4. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | Agenda What is a Data Dictionary? The MySQL Traditional Data Dictionary New Data Dictionary Benefits for users Q & A 1 2 3 4 5
  • 5. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | Data Dictionary What is it
  • 6. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | Data Dictionary Definition • Metadata is information about user data – User table structure – Column definitions – Index definitions – Foreign key definitions – Stored program definitions ... • Data Dictionary collects all metadata in RDBMS
  • 7. CREATE PROCEDURE p1(v INT) SQL SECURITY INVOKER BEGIN ... END Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | Data Dictionary Overview Data Dictionary Table Definitions SP Definitions View Definitions Plugins Time zones Privileges CREATE TABLE customers( id INT AUTO_INCREMENT ... PRIMARY KEY (id), INDEX ... FOREIGN KEY ... )
  • 8. SE InnoDB Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | MySQL Server Data Dictionary Definition 8 The ecosystem Query Executor Optimizer Performance Schema SQL statement Client Parser Result Information Schema Data Dictionary
  • 9. The MySQL Traditional Data Dictionary MySQL 5.6 and earlier Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
  • 10. The MySQL Traditional Data Dictionary • A mix of files and tables: – File based Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | • Tables: FRM • Triggers: TRN, TRG ... – Table based • mysql.time_zone ... • InnoDB has a separate data dictionary
  • 11. The MySQL Traditional Data Dictionary Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | 11 Data Dictionary Files FRM TRG OPT System tables (mysql.) user time_zone proc InnoDB internal data dictionary Archive CSV InnoDB INFORMATION_SCHEMA File Scan Table Scan Intern. Access
  • 12. The MySQL Traditional Data Dictionary • Poor INFORMATION_SCHEMA performance • Makes crash-safe / transactional DDL impossible • Inconsistencies between files and tables • Inconsistencies between DD in InnoDB and the server • File-system dependency (lower-case-table-names) • Makes replication of DDL statements difficult • Too difficult to extend Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | 12 Problems
  • 13. New Data Dictionary A great leap forward Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
  • 14. New Data Dictionary : Main Features Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | • Stored in InnoDB tables • Reliable & crash-safe • Single repository of metadata – for the MySQL server – for Storage Engines – for Plugins • Redundancy • Data Dictionary API • INFORMATION_SCHEMA SQL VIEWs – Queries can be optimized – Improved performance • Metadata versioning • Extendable – Simplify metadata upgrades – Designed with plugins in mind
  • 15. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | New Data Dictionary 15 Transition Data Dictionary Files FRM TRG OPT System tables (mysql.) user time_zone proc InnoDB internal dictionary Archive CSV InnoDB INFORMATION_SCHEMA File Scan SQL DD Table VIEW Table Scan Intern. Access
  • 16. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | New Data Dictionary 16 Overview InnoDB Data Dictionary DD Table User Table INFORMATION SCHEMA Views Archive User Table CSV User Table
  • 17. Plugin Plugin Plugin Tablespace User Table Storage Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | New Data Dictionary InnoDB 17 Architecture Query Executor Parser Optimizer Data Dictionary Tablespace Data Dictionary Internal API Data Dictionary External API Plugin Plugin Archive Engine Storage Engine
  • 18. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | New Data Dictionary • WL#6379: Schema definitions for new DD • InnoDB Data Dictionary Tablespace • Designed with INFORMATION_SCHEMA in mind • Ability to store SE-specific data • Use PK / FK to ensure consistency 18 Data Dictionary tables
  • 19. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | New Data Dictionary API • The only way to access Data Dictionary – For the server core – For Storage Engines – For plugins • Hard to misuse • Internal API (non-stable) and external API (stable) • Provide a way to handle SE specific data 19 Design goals
  • 20. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | New Data Dictionary SDI : Serialized Dictionary Information 20 Redundancy InnoDB Single User TS User Table SDI General TS User Table SDI System TS User Table SDI Data Dictionary Privileges User Table Definition Stored Program
  • 21. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | New Data Dictionary 21 FRM shipping for MySQL Cluster? SELECT ... FROM t1 FSRDMI
  • 22. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | • No FRM files • New INFORMATION_SCHEMA • Migrated to InnoDB: – time zone tables – help tables –mysql.plugins –mysql.servers • Draft of Data Dictionary API • http://labs.mysql.com – Do NOT use it in production – Install on a spare server • MTR can be run 22 New Data Dictionary : Labs Release
  • 23. New Data Dictionary Why does it matter for YOU? Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
  • 24. INFORMATION_SCHEMA performance improvements Get per table size Blog post by Shlomi Noach: http://tinyurl.com/y8cnj7o SELECT TABLE_SCHEMA, TABLE_NAME, ENGINE, SUM(DATA_LENGTH+INDEX_LENGTH) AS size, SUM(INDEX_LENGTH) AS index_size FROM INFORMATION_SCHEMA.TABLES WHERE TABLE_SCHEMA NOT IN ('mysql', 'INFORMATION_SCHEMA') AND ENGINE IS NOT NULL GROUP BY TABLE_SCHEMA, TABLE_NAME; Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | 24 Version Time 5.7.5-m15 0.38 sec Labs Release 0.08 sec
  • 25. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | Case: server crash • There are some data files • No / outdated backup • FRM files lost • How to use those data files? 25 The problem
  • 26. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | Traditional Data Dictionary • “Move FRM files around” • CREATE TABLE t1 (...) • ALTER TABLE t1 DISCARD TABLESPACE • ALTER TABLE t1 IMPORT TABLESPACE ... • Easy to make mistakes New Data Dictionary • Self-descriptive tablespaces (SDI) • Dedicated IMPORT statement • Goal: error-proof procedure 26 Case: server crash
  • 27. For Plugin Developers • A way to access Data Dictionary • Persistent Storage for plugins – Store/restore custom data • Plugins can extend INFORMATION_SCHEMA & PERFORMANCE_SCHEMA – Add new tables – Add new columns to existing tables Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | 27
  • 28. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | Data Dictionary Summary
  • 29. Data Dictionary : Takeaways • Fundamental component in RDBMS • Critical for performance • Critical for reliability • Critical for scalability Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | 29
  • 30. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | The MySQL Traditional Data Dictionary • Mix of files and tables • Server DD and InnoDB DD • Inefficient INFORMATION_SCHEMA • Difficult to extend New Data Dictionary • Crash-safe InnoDB tables • Single repository • INFORMATION_SCHEMA as VIEWs • Designed to be extendable • Aims for backward compatibility • Huge reengineering Data Dictionary : Takeaways
  • 31. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | Questions?
  • 32. MySQL Community Reception @ Oracle OpenWorld Mingle with the MySQL community and the MySQL team from Oracle for a fun and informative evening! • Time: September 30 (Tue) @ 7pm • Jillian’s at Metreon 175 Fourth Street, San Francisco, CA At the corner of Howard and 4th st.; only 2-min walk from Moscone Center Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
  • 33. Oracle University MySQL Training Services Prepare Your Organization to Enable Reliable and High-Performance Web-Based Database Applications Top Courses for Administrators and Developers Top Certifications Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | “Training and team skill have the most significant impact on overall performance of technology and success of technology projects.” - IDC, 2013 Premier Support customers eligible to save 20% on learning credits. Benefits  Expert-led training to support your MySQL learning needs  Flexibility to train in the classroom or online  Hands-on experience to gain real world experience  Key skills needed for database administrators and developers • MySQL for Beginners MySQL for Database Administrators MySQL Performance Tuning MySQL Cluster – NEW - Register Your Interest! MySQL and PHP - Developing Dynamic Web Applications MySQL for Developers MySQL Developer Techniques MySQL 5.6 Database Administrator MySQL 5.6 Developer To find out more about available MySQL Training & Certification offerings, go to: education.oracle.com/mysql RECENTLY RELEASED ALL NEW! MySQL Cluster Training To Register your interest to influence the schedule on this newly released course – go to education.oracle.com/mysql and click on the MySQL Cluster Course
  • 34. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | Thank You! Copyright © 2014, Oracle and/or its affiliates. All rights reserved.