Trafodion

Create table with 128 character-long schema & table names hangs on HortonWorks

Bug #1389791 reported by Weishiun Tsai
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Trafodion
Fix Released
High
Roberta Marton
Trafodion r1.1

Bug Description

On a HortonWorks cluster, the following statement to create a table with a 128 character-long schema and a 128 character-long table hangs. This statement, however, works fine on a Cloudera cluster. This is seen on the v1104_0830 build installed on a 4-node cluster.

Here is the statement to reproduce it:

create table
trafodion.DDL_TAB001_A013_7_CAT1_SCH1_AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA.Y123456789B123456789C123456789D123456789E123456789F123456789G123456789H123456789I123456789J123456789K123456789L123456789M1234567
(a int) no partition;

Tags: sql-cmu
Revision history for this message
Hans Zeller (hans-zeller) wrote :

Just tried this on my development workstation and it also hangs, with this stack trace:

"main" prio=10 tid=0x0000000000bb2800 nid=0x49c8 waiting on condition [0x00007ffffffea000]
   java.lang.Thread.State: TIMED_WAITING (sleeping)
 at java.lang.Thread.sleep(Native Method)
 at org.apache.hadoop.hbase.client.HBaseAdmin.createTable(HBaseAdmin.java:533)
 at org.apache.hadoop.hbase.client.HBaseAdmin.createTable(HBaseAdmin.java:428)
 at org.trafodion.sql.HBaseAccess.HBaseClient.createk(HBaseClient.java:395)

It looks like the region server hangs for such long table names. SeaQuest guaranteed that you could have table names of up to 128 characters. Trafodion has no such guarantee.

The following name worked for me: trafodion.DDL_TAB001_A013_7_CAT1_SCH1_AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA.Y123456789B123456789C123456789D123456789E123456789F123456789G123456789H123456789I123456789

(that's the original name chopped off at the "J".

Renu Varshneya (renu-varshneya)
tags: added: sql-cmu
removed: sql-exe
Paul Low (paul-low-x)
Changed in trafodion:
assignee: nobody → Roberta Marton (roberta-marton)
Paul Low (paul-low-x)
Changed in trafodion:
milestone: none → r1.1
Revision history for this message
Roberta Marton (roberta-marton) wrote :

When Trafodion generates an hbase name, it includes both the catalog and schema name. In this case, the hbase name would look like:

"trafodion.DDL_TAB001_A013_7_CAT1_SCH1_AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA.Y123456789B123456789C123456789D123456789E123456789F123456789G123456789H123456789I123456789J123456789K123456789L123456789M1234567"

Adding up all three parts, the length of the entire name is 268 bytes.
However, the Linux file system only allow names up to 256 bytes.

The following table can be created - total length is 256.

"trafodion.DDL_TAB001_A013_7_CAT1_SCH1_AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA.Y123456789B123456789C123456789D123456789E123456789F123456789G123456789H123456789I123456789J123456789K123456789L12345"

The following table create hangs - total length is 257

"trafodion.DDL_TAB001_A013_7_CAT1_SCH1_AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA.Y123456789B123456789C123456789D123456789E123456789F123456789G123456789H123456789I123456789J123456789K123456789L123456"

Recommend adding a check to make sure the total name length is not longer than what Linux supports.

Changed in trafodion:
status: New → In Progress
Revision history for this message
Roberta Marton (roberta-marton) wrote :

Actually, there was a fairly recent change to only support 255 byte names in HDFS:

https://issues.apache.org/jira/browse/HDFS-6055

Revision history for this message
Roberta Marton (roberta-marton) wrote :

Fixed by adding a length check before asking Hbase to create the table.

Changed in trafodion:
status: In Progress → Fix Committed
Revision history for this message
Weishiun Tsai (wei-shiun-tsai) wrote :

Verified on the v0324 build installed on a workstation. This statement now returns an error as indicated:

>>create table
 trafodion.DDL_TAB001_A013_7_CAT1_SCH1_AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA.Y123456789B123456789C123456789D123456789E123456789F123456789G123456789H123456789I123456789J123456789K123456789L123456789M1234567
+>+> (a int) no partition;

*** ERROR[1030] The HBase name has a length of 267 which is too long. Maximum length supported is 255.

--- SQL operation failed with errors.

Changed in trafodion:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.