doris

Files

Mingyu Chen 7182f14645 [improvement][fix](multi-catalog) speed up list partition prune (#14268 )

In previous implementation, when doing list partition prune, we need to generation `rangeToId`
every time we doing prune.
But `rangeToId` is actually a static data that should be create-once-use-every-where.

So for hive partition, I created the `rangeToId` and all other necessary data structures for partition prunning
in partition cache, so that we can use it directly.

In my test, the cost of partition prune for 10000 partitions reduce from 8s -> 0.2s.

Aslo add "partition" info in explain string for hive table.
```
|   0:VEXTERNAL_FILE_SCAN_NODE                           |
|      predicates: `nation` = '0024c95b'                 |
|      inputSplitNum=1, totalFileSize=4750, scanRanges=1 |
|      partition=1/10000                                 |
|      numNodes=1                                        |
|      limit: 10                                         |
```

Bug fix:
1. Fix bug that es scan node can not filter data
2. Fix bug that query es with predicate like `where substring(test2,2) = "ext2";` will fail at planner phase.
`Unexpected exception: org.apache.doris.analysis.FunctionCallExpr cannot be cast to org.apache.doris.analysis.SlotRef`

TODO:
1. Some problem when quering es version 8: ` Unexpected exception: Index: 0, Size: 0`, will be fixed later.

2022-11-17 08:30:03 +08:00

arm

[docker] ADD Arm Compile Dockerfile (#10600 )

2022-07-27 19:55:42 +08:00

thirdparties

[improvement][fix](multi-catalog) speed up list partition prune (#14268 )

2022-11-17 08:30:03 +08:00

Dockerfile

[doc] update docs (#12615 )

2022-09-15 11:07:34 +08:00

Dockerfile.gcc7

[TLP](step-1) Remove incubator prefix (#10230 )

2022-06-19 19:34:52 +08:00

Dockerfile.gcc10

[TLP](step-1) Remove incubator prefix (#10230 )

2022-06-19 19:34:52 +08:00

README.md

[TLP](step-1) Remove incubator prefix (#10230 )

2022-06-19 19:34:52 +08:00

README.md

Doris Develop Environment based on docker

Preparation

Download the Doris code repo
```
$ cd /to/your/workspace/
$ git clone https://github.com/apache/doris.git
```
You can remove the .git dir in doris/ to make the dir size smaller. So that the following generated docker image can be smaller.

Copy Dockerfile

$ cd /to/your/workspace/
$ cp doris/docker/Dockerfile ./

After preparation, your workspace should like this:

.
├── Dockerfile
├── doris
│   ├── be
│   ├── bin
│   ├── build.sh
│   ├── conf
│   ├── DISCLAIMER-WIP
│   ├── docker
│   ├── docs
│   ├── env.sh
│   ├── fe
│   ├── ...

Build docker image

$ cd /to/your/workspace/
$ docker build -t doris:v1.0  .

doris is docker image repository name and v1.0 is tag name, you can change them to whatever you like.

Use docker image

This docker image you just built does not contain Doris source code repo. You need to download it first and map it to the container. (You can just use the one you used to build this image before)

$ docker run -it -v /your/local/path/doris/:/root/doris/ doris:v1.0
$ docker run -it -v /your/local/.m2:/root/.m2 -v /your/local/doris-DORIS-x.x.x-release/:/root/doris-DORIS-x.x.x-release/ doris:v1.0

Then you can build source code inside the container.

$ cd /root/doris/
$ sh build.sh

NOTICE

The default JDK version is openjdk 11, if you want to use openjdk 8, you can run the command:

$ alternatives --set java java-1.8.0-openjdk.x86_64
$ alternatives --set javac java-1.8.0-openjdk.x86_64
$ export JAVA_HOME=/usr/lib/jvm/java-1.8.0

The version of jdk you used to run FE must be the same version you used to compile FE.

Latest update time

2022-1-23