By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement . We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

If we have a column which is a map of a struct, if the value for the key is null, instead of return null as in spark, hive iceberg will throw a NullPointerException

[2022-03-07 10:46:07] Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Error evaluating struct(_col3,_col2['key'].context).col2 [2022-03-07 10:46:07] at org.apache.hadoop.hive.ql.exec.SelectOperator.process(SelectOperator.java:93) [2022-03-07 10:46:07] at org.apache.hadoop.hive.ql.exec.Operator.baseForward(Operator.java:994) [2022-03-07 10:46:07] at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:940) [2022-03-07 10:46:07] at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:927) [2022-03-07 10:46:07] at org.apache.hadoop.hive.ql.exec.SelectOperator.process(SelectOperator.java:95) [2022-03-07 10:46:07] at org.apache.hadoop.hive.ql.exec.Operator.baseForward(Operator.java:994) [2022-03-07 10:46:07] at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:940) [2022-03-07 10:46:07] at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:927) [2022-03-07 10:46:07] at org.apache.hadoop.hive.ql.exec.FilterOperator.process(FilterOperator.java:126) [2022-03-07 10:46:07] at org.apache.hadoop.hive.ql.exec.Operator.baseForward(Operator.java:994) [2022-03-07 10:46:07] at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:940) [2022-03-07 10:46:07] at org.apache.hadoop.hive.ql.exec.TableScanOperator.process(TableScanOperator.java:125) [2022-03-07 10:46:07] at org.apache.hadoop.hive.ql.exec.MapOperator$MapOpCtx.forward(MapOperator.java:153) [2022-03-07 10:46:07] at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:555) [2022-03-07 10:46:07] ... 20 more [2022-03-07 10:46:07] Caused by: java.lang.NullPointerException [2022-03-07 10:46:07] at org.apache.iceberg.mr.hive.serde.objectinspector.IcebergRecordObjectInspector.getStructFieldData(IcebergRecordObjectInspector.java:75) [2022-03-07 10:46:07] at org.apache.hadoop.hive.ql.exec.ExprNodeFieldEvaluator._evaluate(ExprNodeFieldEvaluator.java:94) [2022-03-07 10:46:07] at org.apache.hadoop.hive.ql.exec.ExprNodeEvaluator.evaluate(ExprNodeEvaluator.java:80) [2022-03-07 10:46:07] at org.apache.hadoop.hive.ql.exec.ExprNodeGenericFuncEvaluator$DeferredExprObject.get(ExprNodeGenericFuncEvaluator.java:88) [2022-03-07 10:46:07] at org.apache.hadoop.hive.ql.udf.generic.GenericUDFStruct.evaluate(GenericUDFStruct.java:70) [2022-03-07 10:46:07] at org.apache.hadoop.hive.ql.exec.ExprNodeGenericFuncEvaluator._evaluate(ExprNodeGenericFuncEvaluator.java:197) [2022-03-07 10:46:07] at org.apache.hadoop.hive.ql.exec.ExprNodeEvaluator.evaluate(ExprNodeEvaluator.java:80) [2022-03-07 10:46:07] at org.apache.hadoop.hive.ql.exec.ExprNodeFieldEvaluator._evaluate(ExprNodeFieldEvaluator.java:79) [2022-03-07 10:46:07] at org.apache.hadoop.hive.ql.exec.ExprNodeEvaluator.evaluate(ExprNodeEvaluator.java:80) [2022-03-07 10:46:07] at org.apache.hadoop.hive.ql.exec.ExprNodeEvaluator.evaluate(ExprNodeEvaluator.java:68) [2022-03-07 10:46:07] at org.apache.hadoop.hive.ql.exec.SelectOperator.process(SelectOperator.java:88) [2022-03-07 10:46:07] ... 33 more

changed the title Hive throw a NullPointerException when accessing a null struct inside a map. Hive : Hive throw a NullPointerException when accessing a null struct inside a map. Mar 7, 2022 If we have a column which is a map of a struct, if the value for the key is null or the key not existing, hive should return null as ask in parent class org.apache.hadoop.hive.serde2.objectinspector.StructObjectInspector and not throwing an NullPointerException If we have a column which is a map of a struct, if the value for the key is null or the key not existing, hive should return null as ask in parent class org.apache.hadoop.hive.serde2.objectinspector.StructObjectInspector and not throwing an NullPointerException If we have a column which is a map of a struct, if the value for the key is null or the key not existing, hive should return null as ask in parent class org.apache.hadoop.hive.serde2.objectinspector.StructObjectInspector and not throwing an NullPointerException If we have a column which is a map of a struct, if the value for the key is null or the key not existing, hive should return null as ask in parent class org.apache.hadoop.hive.serde2.objectinspector.StructObjectInspector and not throwing an NullPointerException If we have a column which is a map of a struct, if the value for the key is null or the key not existing, hive should return null as ask in parent class org.apache.hadoop.hive.serde2.objectinspector.StructObjectInspector and not throwing an NullPointerException If we have a column which is a map of a struct, if the value for the key is null or the key not existing, hive should return null as ask in parent class org.apache.hadoop.hive.serde2.objectinspector.StructObjectInspector and not throwing an NullPointerException Hive : Tez parquet vectorization throw a ClassCastException when accessing a null struct inside a map #4403 If we have a column which is a map of a struct, if the value for the key is null or the key not existing, hive should return null as ask in parent class org.apache.hadoop.hive.serde2.objectinspector.StructObjectInspector and not throwing an NullPointerException