TEZ-4721: Remove legacy Hadoop 2.x support and hadoop-shim modules#505
TEZ-4721: Remove legacy Hadoop 2.x support and hadoop-shim modules#505Aggarwal-Raghav wants to merge 1 commit into
Conversation
|
💔 -1 overall
This message was automatically generated. |
thanks for the patch @Aggarwal-Raghav so far! the same applies to GuavaShim: it was related to incompatible Guava versions, and I'm afraid removing it can make extra work in the future if we find ourselves in a similar situation...but regardless, removing it should be a scope of a different ticket this one can focus on dropping hadoop 2.x support only, and we can make it happen quite easily |
I thought about it before dropping the shims framework, I considered the potential for future incompatibility scenarios. These include the incompatibility between Hadoop3.x and Hadoop 4.x (in the future maybe). But these scenario are speculative and its better to avoid the maintenance of two versions of the hadoop or guava. It is preferable to support the latest version at the present time. We had Hadoop27 profile but if you try to run on master it is not even running becuase of self-referene in 1 of the pom.xml. Another instance is that hadoop2.x is jdk8 but now its jdk21. I hightly doubt even with shims present the JVM 8 and JDK 21 JVM cross-compatiblity will work. But if you think its better to retain shims framework confirm once. I have expressed my thoughts but I don't have the final say here 😅 |
|
ok, I've updated the PR, let me know if its in-line with the scope, I have updated the DefaultHadoopShims to have methods like |
spot-checked, looks good, let me get back to this in few days in detail |
|
Jenkins stopped the last run, will re-trigger. |
…ntation - Retains the core HadoopShim framework and GuavaShim - Removes hadoop27 and hadoop28 profiles from all pom.xml files - Removes hadoop-shim-impls module (Hadoop 2.7 and 2.8 specific shims)
Uh oh!
There was an error while loading. Please reload this page.