MySQL change event producer

The MySql Producer relies on Open Replicator. It uses the Open Replicator to read MySql bin logs and convert them into change events. The Open Replicator project has been forked to https://github.com/Flipkart/open-replicator for the following reasons: * Enable support for MySql version 5.6 Bin Log while the original version supports only version 5.5 event formats. * Support for parsing checksum introduced in version 5.6.

The implementation details of this producer is as follows:

• MySql Producer registers a Replication Listener with Open Replicator. When events are processed in Open Replicator the Replication Listener in MySql producer is given a call back.
• Various Event Processors are registered in MySql producer to handle specific event types (Insert, Update, Delete, InsertV2, UpdateV2, DeleteV2 for DML and various type of DDL events).
• The event processors are called to execute business logic based on the type of event.
  • For events of type DML
    • Transaction Manager is called to handle the event. The Transaction Manager maintains the events and handles the life cycle of the Transaction until the Transaction has been committed.
    • Once the Transaction is committed the Transaction Manager adds the events into Event Buffer.
  • For events of type DDL (Currently for Alter)
    • The AVRO Schema Generator is called to regenerate the Schema for the table and update the schema registry in Relay.
    • This event (DDL) needs to be propagated the Client. This needs to be done. Not yet coded.
• MySql producer generates SCN by a combination of the bin log sql file number and the location with in the bin log.
• The binlog file number is left shifted by 32 bits and the location with in it added to it.
• Example if the bin log file is 1 and the location within the bin log file is 4 (start of bin log file) then SCN would be ((1<<32)+4).