HIVE-29632: hive.fetch.task.caching=true causes unbounded heap allocation thus crashing HiveServer2 with OutOfMemoryError#6510
Open
vikramahuja1001 wants to merge 1 commit into
Conversation
deniskuzZ
reviewed
Jun 1, 2026
| HiveConf.ConfVars.HIVE_FETCH_TASK_CACHING); | ||
| if (cachingEnabled && !AcidUtils.isTransactionalTable(fetch.table)) { | ||
| LOG.debug("Fetch task caching is enabled but table {} is not transactional. " + | ||
| "Caching is only supported for ACID tables to prevent Cleaner race conditions. Disabling.", |
Member
Contributor
Author
There was a problem hiding this comment.
Cleaned the Log statement
vikramahuja1001
force-pushed
the
HIVE-fetch-task-caching-acid-only
branch
from
a77d8aa to
29f35eb
Compare
asf-ci-hive
added
tests pending
tests passed
and removed
tests unstable
tests pending
labels
…eap allocation thus crashing HiveServer2 with OutOfMemoryError
vikramahuja1001
force-pushed
the
HIVE-fetch-task-caching-acid-only
branch
from
29f35eb to
4bce58b
Compare
|
vikramahuja1001
commented
Jun 2, 2026
| "\n" + org.apache.hadoop.util.StringUtils.stringifyException(e)); | ||
| throw new CommandProcessorException(1); | ||
| } finally { | ||
| } catch (RuntimeException e) { |
Contributor
Author
There was a problem hiding this comment.
Reason for this change:
- The SimpleFetchOptimizer change disables caching for non-ACID tables, making FetchTask.execute() a no op. Rows are now fetched lazily via Driver.getResults() after qp.run(). So the exceptions that previously surfaced inside qp.run() now escapes as unchecked RunTimeException from the fetch loop.
- ProcessLocalCmd() only caught IOExceptions, so the RunTImeException bubbled up and never became a CommandProcessorException causing tests tests like, udf_assert_true.q, udf_assert_true2.q to fail.
- The fix adds a separate catch (RuntimeException e) that silently converts it to CommandProcessorException. It intentionally does not reprint the error since FetchTask.executeInner() already did that. Plus doing that will change .q.out file which i was trying to avoid.
Contributor
Author
|
@deniskuzZ , the build is green for this PR. Can you kindly recheck this PR? |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changes were proposed in this pull request?
hive.fetch.task.caching=true causes unbounded heap allocation and crashes HiveServer2 with OutOfMemoryError
Why are the changes needed?
Refer : https://issues.apache.org/jira/browse/HIVE-29632
Does this PR introduce any user-facing change?
No
How was this patch tested?