database/doris
2
0
Fork 0
Code Issues Pull Requests Actions 3 Packages Projects Releases Wiki Activity

[Bug](SchemeChange) Loading tasks during alter job cause modify column failed (#26975)

Browse Source 
When a table is doing schema-change, it adds _doris_shadow prefix in name of modified columns in shadow index.

The writes during schema-change will generate rowset schema with _doris_shadow prefix in BE.

If the alter task arrives at be after the write request, it will use the rowset schema with max version which has the _doris_shadow prefix.

And an error will be thrown as below:

a shadow column is encountered __doris_shadow_p_retailprice
[INTERNAL_ERROR]failed due to operate on shadow column

This commit will disable shadow prefix in rowset meta schema.
This commit is contained in:
xy720
2023-11-17 21:24:53 +08:00
committed by GitHub
parent 0a1a6cf02f
commit e3e249c584
1 changed files with 7 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

7
fe/fe-core/src/main/java/org/apache/doris/planner/OlapTableSink.java
View File

@ -17,6 +17,7 @@
package org.apache.doris.planner;
import org.apache.doris.alter.SchemaChangeHandler;
import org.apache.doris.analysis.Analyzer;
import org.apache.doris.analysis.Expr;
import org.apache.doris.analysis.SlotDescriptor;
@ -252,6 +253,12 @@ public class OlapTableSink extends DataSink {
columns.addAll(indexMeta.getSchema().stream().map(Column::getNonShadowName).collect(Collectors.toList()));
for (Column column : indexMeta.getSchema()) {
TColumn tColumn = column.toThrift();
// When schema change is doing, some modified column has prefix in name. Columns here
// is for the schema in rowset meta, which should be no column with shadow prefix.
// So we should remove the shadow prefix here.
if (column.getName().startsWith(SchemaChangeHandler.SHADOW_NAME_PREFIX)) {
tColumn.setColumnName(column.getNonShadowName());
}
column.setIndexFlag(tColumn, table);
columnsDesc.add(tColumn);
}