hive 使用 show databases 报错

2017-09-05 17:10:41 +08:00
 xiaoheijw

请教各位大神，问题如下：

hive> show databases;
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. org.apache.hadoop.util.NativeCrc32.nativeComputeChunkedSumsByteArray(II[BI[BIILjava/lang/String;JZ)V

谷歌到一个一样的错误，但是作者的回答不懂是什么，说 native 的问题。不知道 native 是什么 答案链接： https://ask.hellobi.com/question/21542 错误日志如下：

2017-09-05T08:48:21,374 DEBUG [4dcf2a35-1dad-4899-bcab-f25a85a6778c main] ql.Driver: Acquired the compile lock.
2017-09-05T08:48:21,375 DEBUG [main] conf.VariableSubstitution: Substitution is on: show databases
2017-09-05T08:48:21,376  INFO [4dcf2a35-1dad-4899-bcab-f25a85a6778c main] ql.Driver: Compiling command(queryId=root_20170905084821_10312949-18c9-439a-9573-015e66e827fe): show databases
2017-09-05T08:48:21,378 DEBUG [4dcf2a35-1dad-4899-bcab-f25a85a6778c main] parse.ParseDriver: Parsing command: show databases
2017-09-05T08:48:21,380 DEBUG [4dcf2a35-1dad-4899-bcab-f25a85a6778c main] parse.ParseDriver: Parse Completed
2017-09-05T08:48:21,402  INFO [4dcf2a35-1dad-4899-bcab-f25a85a6778c main] ql.Driver: Semantic Analysis Completed
2017-09-05T08:48:21,404 DEBUG [4dcf2a35-1dad-4899-bcab-f25a85a6778c main] lazy.LazySerDeParameters: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe initialized with: columnNames=[database_name] columnTypes=[string] separator=[[B@70d63e05] nullstring=  lastColumnTakesRest=false timestampFormats=null
2017-09-05T08:48:21,405  INFO [4dcf2a35-1dad-4899-bcab-f25a85a6778c main] ql.Driver: Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:database_name, type:string, comment:from deserializer)], properties:null)
2017-09-05T08:48:21,409 DEBUG [4dcf2a35-1dad-4899-bcab-f25a85a6778c main] lazy.LazySerDeParameters: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe initialized with: columnNames=[database_name] columnTypes=[string] separator=[[B@485e13d7] nullstring=  lastColumnTakesRest=false timestampFormats=null
2017-09-05T08:48:21,410  INFO [4dcf2a35-1dad-4899-bcab-f25a85a6778c main] exec.ListSinkOperator: Initializing operator LIST_SINK[0]
2017-09-05T08:48:21,411 DEBUG [4dcf2a35-1dad-4899-bcab-f25a85a6778c main] lazy.LazySerDeParameters: org.apache.hadoop.hive.serde2.DelimitedJSONSerDe initialized with: columnNames=[] columnTypes=[] separator=[[B@43984213] nullstring=  lastColumnTakesRest=false timestampFormats=null
2017-09-05T08:48:21,411 DEBUG [4dcf2a35-1dad-4899-bcab-f25a85a6778c main] exec.ListSinkOperator: Initialization Done 0 LIST_SINK
2017-09-05T08:48:21,412 DEBUG [4dcf2a35-1dad-4899-bcab-f25a85a6778c main] exec.ListSinkOperator: Operator 0 LIST_SINK initialized
2017-09-05T08:48:21,412 DEBUG [4dcf2a35-1dad-4899-bcab-f25a85a6778c main] exec.ListSinkOperator: Initialization Done 0 LIST_SINK done is reset.
2017-09-05T08:48:21,412  INFO [4dcf2a35-1dad-4899-bcab-f25a85a6778c main] metadata.Hive: Dumping metastore api call timing information for : compilation phase
2017-09-05T08:48:21,412 DEBUG [4dcf2a35-1dad-4899-bcab-f25a85a6778c main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(HiveConf, )=0, flushCache_()=9}
2017-09-05T08:48:21,413  INFO [4dcf2a35-1dad-4899-bcab-f25a85a6778c main] ql.Driver: Completed compiling command(queryId=root_20170905084821_10312949-18c9-439a-9573-015e66e827fe); Time taken: 0.037 seconds
2017-09-05T08:48:21,413  INFO [4dcf2a35-1dad-4899-bcab-f25a85a6778c main] ql.Driver: Concurrency mode is disabled, not creating a lock manager
2017-09-05T08:48:21,414  INFO [4dcf2a35-1dad-4899-bcab-f25a85a6778c main] ql.Driver: Executing command(queryId=root_20170905084821_10312949-18c9-439a-9573-015e66e827fe): show databases
2017-09-05T08:48:21,415  INFO [4dcf2a35-1dad-4899-bcab-f25a85a6778c main] ql.Driver: Starting task [Stage-0:DDL] in serial mode
2017-09-05T08:48:21,449 DEBUG [4dcf2a35-1dad-4899-bcab-f25a85a6778c main] sqlstd.SQLStdHiveAuthorizationValidator: Obtained following objects in  filterListCmdObjects [Object [type=DATABASE, name=default], Object [type=DATABASE, name=resultdb], Object [type=DATABASE, name=test]] for user root. Context Info: QueryContext [commandString=null, forwardedAddresses=null]
2017-09-05T08:48:21,450  INFO [4dcf2a35-1dad-4899-bcab-f25a85a6778c main] exec.DDLTask: results : 3
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. org.apache.hadoop.util.NativeCrc32.nativeComputeChunkedSumsByteArray(II[BI[BIILjava/lang/String;JZ)V
2017-09-05T08:48:21,468 ERROR [4dcf2a35-1dad-4899-bcab-f25a85a6778c main] exec.DDLTask: java.lang.UnsatisfiedLinkError: org.apache.hadoop.util.NativeCrc32.nativeComputeChunkedSumsByteArray(II[BI[BIILjava/lang/String;JZ)V
	at org.apache.hadoop.util.NativeCrc32.nativeComputeChunkedSumsByteArray(Native Method)
	at org.apache.hadoop.util.NativeCrc32.calculateChunkedSumsByteArray(NativeCrc32.java:86)
	at org.apache.hadoop.util.DataChecksum.calculateChunkedSums(DataChecksum.java:430)
	at org.apache.hadoop.fs.FSOutputSummer.writeChecksumChunks(FSOutputSummer.java:202)
	at org.apache.hadoop.fs.FSOutputSummer.flushBuffer(FSOutputSummer.java:163)
	at org.apache.hadoop.fs.FSOutputSummer.flushBuffer(FSOutputSummer.java:144)
	at org.apache.hadoop.fs.ChecksumFileSystem$ChecksumFSOutputSummer.close(ChecksumFileSystem.java:412)
	at org.apache.hadoop.fs.FSDataOutputStream$PositionCache.close(FSDataOutputStream.java:72)
	at org.apache.hadoop.fs.FSDataOutputStream.close(FSDataOutputStream.java:106)
	at org.apache.hadoop.io.IOUtils.cleanup(IOUtils.java:244)
	at org.apache.hadoop.io.IOUtils.closeStream(IOUtils.java:261)
	at org.apache.hadoop.hive.ql.exec.DDLTask.showDatabases(DDLTask.java:2515)
	at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:439)
	at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:199)
	at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:100)
	at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:2182)
	at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1838)
	at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1525)
	at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1236)
	at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1226)
	at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:233)
	at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:184)
	at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:403)
	at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:821)
	at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:759)
	at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:686)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
	at org.apache.hadoop.util.RunJar.main(RunJar.java:136)

2017-09-05T08:48:21,469 ERROR [4dcf2a35-1dad-4899-bcab-f25a85a6778c main] ql.Driver: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. org.apache.hadoop.util.NativeCrc32.nativeComputeChunkedSumsByteArray(II[BI[BIILjava/lang/String;JZ)V
2017-09-05T08:48:21,469 DEBUG [4dcf2a35-1dad-4899-bcab-f25a85a6778c main] ql.Driver: Shutting down query show databases
2017-09-05T08:48:21,469  INFO [4dcf2a35-1dad-4899-bcab-f25a85a6778c main] metadata.Hive: Dumping metastore api call timing information for : execution phase
2017-09-05T08:48:21,469 DEBUG [4dcf2a35-1dad-4899-bcab-f25a85a6778c main] metadata.Hive: Total time spent in each metastore function (ms): {isCompatibleWith_(HiveConf, )=0, getAllDatabases_()=34}
2017-09-05T08:48:21,470  INFO [4dcf2a35-1dad-4899-bcab-f25a85a6778c main] ql.Driver: Completed executing command(queryId=root_20170905084821_10312949-18c9-439a-9573-015e66e827fe); Time taken: 0.056 seconds
2017-09-05T08:48:21,470 DEBUG [4dcf2a35-1dad-4899-bcab-f25a85a6778c main] ql.Driver: Shutting down query show databases
2017-09-05T08:48:21,470 DEBUG [4dcf2a35-1dad-4899-bcab-f25a85a6778c main] exec.ListSinkOperator: close called for operator LIST_SINK[0]
2017-09-05T08:48:21,470  INFO [4dcf2a35-1dad-4899-bcab-f25a85a6778c main] exec.ListSinkOperator: Closing operator LIST_SINK[0]
2017-09-05T08:48:21,471 DEBUG [4dcf2a35-1dad-4899-bcab-f25a85a6778c main] exec.ListSinkOperator: 0 Close done
2017-09-05T08:48:21,471 DEBUG [4dcf2a35-1dad-4899-bcab-f25a85a6778c main] ql.Context: Deleting result file: file:/tmp/hive/local/4dcf2a35-1dad-4899-bcab-f25a85a6778c/hive_2017-09-05_08-48-21_377_4100251675857995130-1/-local-10000
2017-09-05T08:48:21,473 DEBUG [4dcf2a35-1dad-4899-bcab-f25a85a6778c main] ql.Context: Deleting scratch dir: file:/tmp/hive/local/4dcf2a35-1dad-4899-bcab-f25a85a6778c/hive_2017-09-05_08-48-21_377_4100251675857995130-1
1731 次点击
所在节点    问与答
0 条回复

这是一个专为移动设备优化的页面（即为了让你能够在 Google 搜索结果里秒开这个页面），如果你希望参与 V2EX 社区的讨论，你可以继续到 V2EX 上打开本讨论主题的完整版本。

https://www.v2ex.com/t/388371

V2EX 是创意工作者们的社区，是一个分享自己正在做的有趣事物、交流想法，可以遇见新朋友甚至新机会的地方。

V2EX is a community of developers, designers and creative people.

© 2021 V2EX