CCLOG-2401: Support for hierarchical ORC data and logical tpes.

[snehashisp]

Problem

Orc support in the current connector has several limitations.

• Currently Support for orc is limited to non-hierarchical or flattened struct data. Nested structs are not properly supported.
• Logical types (Date, Timestamp, Time, Decimal) are not correctly inferred to their corresponding hive types. Instead they are inferred as their base type (int32, int64 and bytes[]).
• Arrays and Maps were using ArrayPrimitiveWritable and MapWritable respectively which failed as the OrcStruct requires native java List and Map types for for array and Map type data. The conversion process of arrays and maps also did not support structured subtypes and nested arrays.

Solution

Some of the problems were identified and a solution was provided in #636. This is an extension of the work done there.

• OrcUtil is refactored and improved to support hierarchical Orc data.
• Support for parsing logical types correctly is added in common. This will be pulled in here after merge. The required changes in OrcUtil for writing logical types are added.
• Arrays and Maps are written correctly.
• Unit Tests and Integration tests are added.
• As part of the solution OrcFileReader needed to be updated to be able to recover the full hierarchical connect schema.

Does this solution apply anywhere else?

• yes
• no

If yes, where?

Test Strategy

Testing done:

• Unit tests
• Integration tests
• System tests
• Manual tests

Release Plan

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.
1 out of 2 committers have signed the CLA.

✅ snehashisp
❌ jsims-slower
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[sudeshwasnik]

lgtm! thanks @snehashisp !