mxosrvr core in SRVR::FETCH2bulk when columns in particular order

Vito did more testing and found more repo table queries that cause mxosrvr cores:

select EXEC_START_UTC_TS,QUERY_ID,USER_NAME,QUERY_TEXT from TRAFODION."_REPOS_".METRIC_QUERY_TABLE;
select EXEC_START_UTC_TS,QUERY_ID,USER_NAME,EXPLAIN_PLAN from TRAFODION."_REPOS_".METRIC_QUERY_TABLE;
select EXEC_START_UTC_TS,QUERY_ID,ROLE_NAME,QUERY_TEXT from TRAFODION."_REPOS_".METRIC_QUERY_TABLE;
select EXEC_START_UTC_TS,QUERY_ID,ROLE_NAME,EXPLAIN_PLAN from TRAFODION."_REPOS_".METRIC_QUERY_TABLE;
select EXEC_START_UTC_TS,QUERY_ID,SESSION_ID,QUERY_TEXT from TRAFODION."_REPOS_".METRIC_QUERY_TABLE;
select EXEC_START_UTC_TS,QUERY_ID,SESSION_ID,EXPLAIN_PLAN from TRAFODION."_REPOS_".METRIC_QUERY_TABLE;
select EXEC_START_UTC_TS,QUERY_ID,STATEMENT_ID,QUERY_TEXT from TRAFODION."_REPOS_".METRIC_QUERY_TABLE;
select EXEC_START_UTC_TS,QUERY_ID,STATEMENT_ID,EXPLAIN_PLAN from TRAFODION."_REPOS_".METRIC_QUERY_TABLE;
select EXEC_START_UTC_TS,QUERY_ID,STATEMENT_TYPE,QUERY_TEXT from TRAFODION."_REPOS_".METRIC_QUERY_TABLE;
select EXEC_START_UTC_TS,QUERY_ID,STATEMENT_TYPE,EXPLAIN_PLAN from TRAFODION."_REPOS_".METRIC_QUERY_TABLE;
select EXEC_START_UTC_TS,QUERY_ID,STATEMENT_SUBTYPE,QUERY_TEXT from TRAFODION."_REPOS_".METRIC_QUERY_TABLE;
select EXEC_START_UTC_TS,QUERY_ID,STATEMENT_SUBTYPE,EXPLAIN_PLAN from TRAFODION."_REPOS_".METRIC_QUERY_TABLE;
select EXEC_START_UTC_TS,QUERY_ID,QUERY_STATUS,QUERY_TEXT from TRAFODION."_REPOS_".METRIC_QUERY_TABLE;
select EXEC_START_UTC_TS,QUERY_ID,QUERY_SUB_STATUS,QUERY_TEXT from TRAFODION."_REPOS_".METRIC_QUERY_TABLE;

Same behavior observed on an upgraded system and clean install.

This seems to be a case of memory corruption, which happens right after SQL_EXEC_Fetch() is called. There are a couple of things that seems to trigger this.

1. Executing a query like below where the query_text column is the last in the select list.
select EXEC_START_UTC_TS,QUERY_ID,USER_NAME,QUERY_TEXT from TRAFODION."_REPOS_".METRIC_QUERY_TABLE;

AND

2. The number of rows in the table is greater than or equal to 100.

The corruption occurs along the fetch buffer that is passed to SQL via WSQL_EXEC_SetDescItem() with desc type of SQLDESC_ROWWISE_ROWSET_PTR. The first byte right after the fetch buffer ends seems to be overwritten with a null character. If that particular byte happens to be already allocated and in use then it would cause a core. Below is an output from a gdb session.

(gdb) p pSrvrStmt->outputDescVarBuffer[(pSrvrStmt->maxRowCnt*pSrvrStmt->outputDescVarBufferLen)+1]
$81 = 28 '\034'

6157 retcode = WSQL_EXEC_Fetch(pStmt, pDesc, 0);
(gdb)
[New Thread 0x7fffd1ae5700 (LWP 25017)]
6158 if (retcode < SQL_SUCCESS)

(gdb) p pSrvrStmt->outputDescVarBuffer[(pSrvrStmt->maxRowCnt*pSrvrStmt->outputDescVarBufferLen)+1]
$82 = 0 '\000'

The pSrvrStmt->outputDescVarBuffer is allocated for pSrvrStmt->maxRowCnt*pSrvrStmt->outputDescVarBufferLen.

As can be seen above the byte right after where the buffer ends had a valid value '\034' before the call to fetch and after the call that byte is overwritten by a null character.

Note, this issue does not occur if the query_text column is not the last in the select list even though the table may have > 100 rows.

Since the memory corruption is happening in the CLI call assigning this bug the SQL executor group.

what is the datatype of query_text in the calling app?
Is it null terminated varchar?
what happens if you turn off rowsets and retrieve
as non-rowset fetches?

The calculation of the rowset row sizze needs to be looked at in connectivity. Seems like it's lower than needed.

Row length being moved : 200436

So here is what should be allocated :

11 + (timestamp)
160 + (query_id
  = 1 byte alignment
2+256 + (null indicator +username)

2+4+200000 (null indicator + VC indicator length + query_text length)

Instead we are allocating is 200434 for each row in conn side

Fix proposed to branch: master
Review: https://review.trafodion.org/1524

Reviewed: https://review.trafodion.org/1524
Committed: https://github.com/trafodion/core/commit/7352e31841c1aac60373e0939fdeb493c6e07f76
Submitter: Trafodion Jenkins
Branch: master

commit 7352e31841c1aac60373e0939fdeb493c6e07f76
Author: Tharaknath Capirala <email address hidden>
Date: Fri Apr 17 06:09:17 2015 +0000

    Fix for bug 1438775

    The fetch buffer size calculation did not account for varchar indicator
    length for columns greater than 32K. The indicator length in this case is 4 bytes instead of 2. Currently, the buffer length calculation was using 2, which resulted in allocating insufficient memory leading to a corruption.

    Fixes bug 1438775

    Change-Id: Ib16b6644ca3c7f36d96687a33ea36ad4f0ffe903

we are not seeing these cores now, tested with rc1.