database/doris
2
0
Fork 0
Code Issues Pull Requests Actions 3 Packages Projects Releases Wiki Activity
Files
511adb8ea39f0193c2cd53c5b798ad398ab1958f
doris/docker
History
Thearas baaf026e82 [fix](hive docker)Reserve host port for hive2 namenode and datanode (#47262) (#47354) 
Problem Summary:

The [External hive

CI](http://43.132.222.7:8111/buildConfiguration/Doris_External_Regression/612304?buildTab=log&linesState=3650&logView=flowAware)
failed because of `namenode` error( 50070 port already in used), docker
logs:
```txt
2025-01-21T04:22:37.955682469Z java.net.BindException: Port in use: 0.0.0.0:50070
2025-01-21T04:22:37.955686106Z 	at org.apache.hadoop.http.HttpServer2.openListeners(HttpServer2.java:940)
2025-01-21T04:22:37.955689402Z 	at org.apache.hadoop.http.HttpServer2.start(HttpServer2.java:876)
2025-01-21T04:22:37.955692708Z 	at org.apache.hadoop.hdfs.server.namenode.NameNodeHttpServer.start(NameNodeHttpServer.java:142)
2025-01-21T04:22:37.955697828Z 	at org.apache.hadoop.hdfs.server.namenode.NameNode.startHttpServer(NameNode.java:760)
2025-01-21T04:22:37.955701444Z 	at org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:639)
2025-01-21T04:22:37.955704831Z 	at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:819)
2025-01-21T04:22:37.955708237Z 	at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:803)
2025-01-21T04:22:37.955711674Z 	at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1500)
2025-01-21T04:22:37.955715090Z 	at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1566)
2025-01-21T04:22:37.955718446Z Caused by: java.net.BindException: Address already in use
2025-01-21T04:22:37.955722013Z 	at sun.nio.ch.Net.bind0(Native Method)
2025-01-21T04:22:37.955725460Z 	at sun.nio.ch.Net.bind(Net.java:433)
2025-01-21T04:22:37.955729227Z 	at sun.nio.ch.Net.bind(Net.java:425)
2025-01-21T04:22:37.955733074Z 	at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:223)
2025-01-21T04:22:37.955736600Z 	at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74)
2025-01-21T04:22:37.955740197Z 	at org.mortbay.jetty.nio.SelectChannelConnector.open(SelectChannelConnector.java:216)
2025-01-21T04:22:37.955743884Z 	at org.apache.hadoop.http.HttpServer2.openListeners(HttpServer2.java:934)
2025-01-21T04:22:37.955747391Z 	... 8 more
2025-01-21T04:22:37.961686454Z 25/01/21 04:22:37 INFO util.ExitUtil: Exiting with status 1
```

The best choice is avoid the services using server port at range
`/proc/sys/net/ipv4/ip_local_port_range` (32768-60999). But since the
namenode [hardcode exposing port `50070` in docker
image](https://hub.docker.com/layers/bde2020/hadoop-datanode/2.0.0-hadoop2.7.4-java8/images/sha256-5623fca5e36d890983cdc6cfd29744d1d65476528117975b3af6a80d99b3c62f),
so we add the port to `net.ipv4.ip_local_reserved_ports` and introduce a
new flags `--reserve-ports` to control it (default false, because not
everyone want to modify system reserved ports).

Change-Id: I03a81e9931cb555695199436b6f0517cccf83588
2025-01-24 16:12:03 +08:00
..
compilation
[fix](dockerfile) Switch repos to point to to vault.centos.org because CentOS 7 is EOL (#37568) (#37763)
2024-07-15 15:57:56 +08:00
runtime
[typo](docker) Adjust the indentation format of the init_be and entry_point scripts, as well as the duration of loop execution(Merge-2.1). (#45860)
2024-12-25 12:16:42 +08:00
thirdparties
[fix](hive docker)Reserve host port for hive2 namenode and datanode (#47262) (#47354)
2025-01-24 16:12:03 +08:00
README.md
[chore](orc) Update orc lib to third party lib(1.8.3) using git submodule. (#18531)
2023-04-12 10:37:50 +08:00

README.md

Doris Develop Environment based on docker

Preparation

  1. Download the Doris code repo

    $ cd /to/your/workspace/
$ git clone https://github.com/apache/doris.git
$ cd doris
$ git submodule update --init --recursive

    You can remove the .git dir in doris/ to make the dir size smaller. So that the following generated docker image can be smaller.

  2. Copy Dockerfile

    $ cd /to/your/workspace/
$ cp doris/docker/Dockerfile ./

After preparation, your workspace should like this:

.
├── Dockerfile
├── doris
│   ├── be
│   ├── bin
│   ├── build.sh
│   ├── conf
│   ├── DISCLAIMER-WIP
│   ├── docker
│   ├── docs
│   ├── env.sh
│   ├── fe
│   ├── ...

Build docker image

$ cd /to/your/workspace/
$ docker build -t doris:v1.0  .

doris is docker image repository name and v1.0 is tag name, you can change them to whatever you like.

Use docker image

This docker image you just built does not contain Doris source code repo. You need to download it first and map it to the container. (You can just use the one you used to build this image before)

$ docker run -it -v /your/local/path/doris/:/root/doris/ doris:v1.0
$ docker run -it -v /your/local/.m2:/root/.m2 -v /your/local/doris-DORIS-x.x.x-release/:/root/doris-DORIS-x.x.x-release/ doris:v1.0

Then you can build source code inside the container.

$ cd /root/doris/
$ sh build.sh

NOTICE

The default JDK version is openjdk 11, if you want to use openjdk 8, you can run the command:

$ alternatives --set java java-1.8.0-openjdk.x86_64
$ alternatives --set javac java-1.8.0-openjdk.x86_64
$ export JAVA_HOME=/usr/lib/jvm/java-1.8.0

The version of jdk you used to run FE must be the same version you used to compile FE.

Latest update time

2022-1-23