Bigquery insertAll operation is checking for a field in the inserted Row which is not present in the schema. I have set setIgnoreUnknownValues to true and also set setSkipInvalidRows to true.
Here is the exception Trace,
java.lang.NullPointerException: null value in entry: xyz=null
at com.google.common.collect.CollectPreconditions.checkEntryNotNull(CollectPreconditions.java:33) ~[bqstream.jar:?]
at com.google.common.collect.RegularImmutableMap.fromEntryArray(RegularImmutableMap.java:69) ~[bqstream.jar:?]
at com.google.common.collect.RegularImmutableMap.fromEntries(RegularImmutableMap.java:46) ~[bqstream.jar:?]
at com.google.common.collect.ImmutableMap.copyOf(ImmutableMap.java:354) ~[bqstream.jar:?]
at com.google.common.collect.ImmutableMap.copyOf(ImmutableMap.java:327) ~[bqstream.jar:?]
at com.google.cloud.bigquery.InsertAllRequest$RowToInsert.<init>(InsertAllRequest.java:81) ~[bqstream.jar:?]
at com.google.cloud.bigquery.InsertAllRequest$RowToInsert.of(InsertAllRequest.java:141) ~[bqstream.jar:?]
at com.XXXXXXXXXX.bqstream.data.Data.lambda$getData$0(Data.java:46) ~[bqstream.jar:?]
at java.util.ArrayList.forEach(Unknown Source) ~[?:1.8.0_141]
at com.XXXXXXXXXX.bqstream.data.Data.getData(Data.java:34) [bqstream.jar:?]
at com.XXXXXXXXXX.bqstream.BatchProcessor.apply(BatchProcessor.java:22) [bqstream.jar:?]
at com.XXXXXXXXXX.mdaolib.controller.BatchOutHandler.run(BatchOutHandler.java:35) [bqstream.jar:?]
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) [?:1.8.0_141]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) [?:1.8.0_141]
at java.lang.Thread.run(Unknown Source) [?:1.8.0_141]After looking at the code for InsertAll, it seems like some Preconditons call is performed on the row to be inserted. But why is the same performed on a data not provided in the Schema.
Bigquery insertAll operation is checking for a field in the inserted Row which is not present in the schema. I have set setIgnoreUnknownValues to true and also set setSkipInvalidRows to true.
Here is the exception Trace,
After looking at the code for InsertAll, it seems like some Preconditons call is performed on the row to be inserted. But why is the same performed on a data not provided in the Schema.