Показаны сообщения с ярлыком Hive. Показать все сообщения
Показаны сообщения с ярлыком Hive. Показать все сообщения
, ,

Permission denied when loading data into hive table

вторник, 8 октября 2013 г. 0 коммент.

I got an error:


[root ~]# sudo -u hive hive
hive> LOAD DATA INPATH '/user/hdfs/d_product_type_hive_exp.csv' OVERWRITE INTO TABLE D_PRODUCT_TYPE_HIVE;
Loading data to table default.d_product_type_hive
rmr: DEPRECATED: Please use 'rm -r' instead.
Moved: 'hdfs://xxxxx:8020/user/hive/warehouse/d_product_type_hive' to trash at: hdfs://xxxxx:8020/user/hive/.Trash/Current
Failed with exception Permission denied: user=hive, access=WRITE, inode="/user/hdfs":hdfs:supergroup:drwxr-xr-x
at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:224)
at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:204)
at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:152)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkPermission(FSNamesystem.java:4716)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkPermission(FSNamesystem.java:4698)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkParentAccess(FSNamesystem.java:4666)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.renameToInternal(FSNamesystem.java:2698)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.renameToInt(FSNamesystem.java:2666)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.renameTo(FSNamesystem.java:2645)
at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.rename(NameNodeRpcServer.java:589)
at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.rename(ClientNamenodeProtocolServerSideTranslatorPB.java:381)
at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java:44964)
at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:453)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1002)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1701)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1697)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1408)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1695)

FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.MoveTask
hive>

Cause: Command "LOAD DATA"(without LOCAL) moves the data to the final location and the user hive don't have the rights to delete the file.
Action: Give the right to "write" on the directory with the file.


[root ~]# sudo -u hdfs hadoop fs -ls -d /user/hdfs/
Found 1 items
drwxr-xr-x - hdfs supergroup 0 2013-10-08 18:28 /user/hdfs
[root ~]# sudo -u hdfs hadoop fs -chmod o+w /user/hdfs/
[root ~]# sudo -u hdfs hadoop fs -ls -d /user/hdfs/
Found 1 items
drwxr-xrwx - hdfs supergroup 0 2013-10-08 18:28 /user/hdfs

Try again:


[root ~]# sudo -u hive hive
hive> LOAD DATA INPATH '/user/hdfs/d_product_type_hive_exp.csv' OVERWRITE INTO TABLE D_PRODUCT_TYPE_HIVE;
Loading data to table default.d_product_type_hive
rmr: DEPRECATED: Please use 'rm -r' instead.
Moved: 'hdfs://xxxxx:8020/user/hive/warehouse/d_product_type_hive' to trash at: hdfs://xxxxx:8020/user/hive/.Trash/Current
Table default.d_product_type_hive stats: [num_partitions: 0, num_files: 1, num_rows: 0, total_size: 354, raw_data_size: 0]
OK
Time taken: 5.638 seconds
hive>
Читать полностью

,

Hive: mismatched input '<EOF>' expecting StringLiteral near 'BY' in table row format's field separator

понедельник, 7 октября 2013 г. 3 коммент.

I got an error “FAILED: ParseException line 1:155 mismatched input '<EOF>' expecting StringLiteral near 'BY' in table row format's field separator” when created table in hive.


hive> CREATE TABLE IF NOT EXISTS D_PRODUCT_TYPE_HIVE (PROD_TYPE_ID DOUBLE, PROD_TYPE_NAME STRING, DESCRIPTION STRING) ROW FORMAT DELIMITED FIELDS TERMINATED BY ';' LINES TERMINATED BY '\n';
FAILED: ParseException line 1:155 mismatched input '< EOF>' expecting StringLiteral near 'BY' in table row format's field separator
hive>

Semi-colon is reserved character. It needs to be escaped, '\;' or octal code '\073', but not hex code '\03B'.



hive> CREATE TABLE IF NOT EXISTS D_PRODUCT_TYPE_HIVE (PROD_TYPE_ID DOUBLE, PROD_TYPE_NAME STRING, DESCRIPTION STRING) ROW FORMAT DELIMITED FIELDS TERMINATED BY '\;' LINES TERMINATED BY '\n';
OK
Time taken: 0.308 seconds
hive>
Читать полностью