# Apache Doris 发布流程 Apache 的发布必须至少是 IPMC 成员，拥有 apache 邮箱的commiter，这个角色叫做 release manager。 发布的大致流程如下： 1. 在社区发起 DISCUSS; 2. 准备分支和打 tag; 3. 将 tag 打包签名； 4. 上传签名的软件包到 Apache SVN 的 DEV 目录 3. 发社区投票邮件 4. 投票通过后，发 Result 邮件 5. 发邮件到 general@incubator.apache.org 进行投票 6. 发 Result 邮件到 general@incubator.apache.org 7. 上传签名的软件包到 Apache SVN 的 release 目录，并生成相关链接 8. 准备 release note 并发 Announce 邮件到 general@incubator.apache.org 9. 在 Doris 官网和 github 发布下载链接 Release manager 在发布前需要先生成自己的签名公钥，并上传到公钥服务器，之后就可以用这个公钥对准备发布的软件包进行签名。 ## 1. 准备发布 ### 1.1 在社区发起 DISCUSS 如果觉得已经修复了很多bug，开发了比较重要的 feature，任何 IPMC 成员都可以发起 DISCUSS 讨论发布新版本。 可以发起一个标题为 [DISCUSS] x.y.z release 的邮件，在社区内部进行讨论，说明已经修复了哪些bug，开发了哪些 features。 如果 DISCUSS 邮件得到大家支持就可以进行下一步。 ### 1.2 准备分支 发布前需要先新建一个分支，这个分支要进行比较充分的测试，使得功能可用，bug收敛，重要bug都得到修复。 例如： ``` $ git checkout -b branch-0.9 ``` ### 1.3 打 tag 当上述分支已经比较稳定后，就可以在此分支上打 tag。 记得在创建 tag 时，修改 `gensrc/script/gen_build_version.sh` 中的 `build_version` 变量。如 `build_version="0.10.0-release"` 例如： ``` $ git checkout branch-0.9 $ git tag -a 0.9.0-rc01 -m "0.9.0 release candidate 01" $ git push origin 0.9.0-rc01 Counting objects: 1, done. Writing objects: 100% (1/1), 165 bytes | 0 bytes/s, done. Total 1 (delta 0), reused 0 (delta 0) To git@github.com:apache/incubator-doris.git * [new tag] 0.9.0-rc01 -> 0.9.0-rc01 $ git tag ``` ## 2. 签名软件 GnuPG 的安装配置 ### 2.1 GnuPG 1991年，程序员Phil Zimmermann为了避开政府监视，开发了加密软件PGP。这个软件非常好用，迅速流传开来，成了许多程序员的必备工具。但是，它是商业软件，不能自由使用。所以，自由软件基金会决定，开发一个PGP的替代品，取名为GnuPG。这就是GPG的由来。 ### 2.2 安装配置 CentOS 安装命令： ``` yum install gnupg ``` 安装完成后，默认配置文件 gpg.conf 会放在 home 目录下。 ``` ~/.gnupg/gpg.conf ``` 如果不存在这个目录或文件，可以直接创建一个空文件。 编辑gpg.conf, 修改或者增加 keyserver 配置： ``` keyserver hkp://keys.gnupg.net ``` Apache 签名推荐 SHA512， 可以通过配置 gpg 完成。 编辑gpg.conf, 增加下面的三行： ``` personal-digest-preferences SHA512 cert-digest-algo SHA512 default-preference-list SHA512 SHA384 SHA256 SHA224 AES256 AES192 AES CAST5 ZLIB BZIP2 ZIP Uncompressed ``` ## 3. 生成新的签名 ### 3.1 准备签名 推荐的生成新签名的设置： 这里必须通过 SecureCRT 等终端直接登录用户账户，不能通过 su - user 或者 ssh 转，否则密码输入 box 会显示不出来而报错。 先看下 gpg 的 version 以及是否支持 SHA512. ``` $ gpg --version gpg (GnuPG) 2.0.22 libgcrypt 1.5.3 Copyright (C) 2013 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Home: ~/.gnupg Supported algorithms: Pubkey: RSA, ?, ?, ELG, DSA Cipher: IDEA, 3DES, CAST5, BLOWFISH, AES, AES192, AES256, TWOFISH, CAMELLIA128, CAMELLIA192, CAMELLIA256 Hash: MD5, SHA1, RIPEMD160, SHA256, SHA384, SHA512, SHA224 Compression: Uncompressed, ZIP, ZLIB, BZIP2 ``` ### 3.2 生成新的签名 ``` $ gpg --gen-key gpg (GnuPG) 2.0.22; Copyright (C) 2013 Free Software Foundation, Inc. This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Please select what kind of key you want: (1) RSA and RSA (default) (2) DSA and Elgamal (3) DSA (sign only) (4) RSA (sign only) Your selection? 1 RSA keys may be between 1024 and 4096 bits long. What keysize do you want? (2048) 4096 Requested keysize is 4096 bits Please specify how long the key should be valid. 0 = key does not expire = key expires in n days w = key expires in n weeks m = key expires in n months y = key expires in n years Key is valid for? (0) Key does not expire at all Is this correct? (y/N) y GnuPG needs to construct a user ID to identify your key. Real name: xxx Name must be at least 5 characters long Real name: xxx-yyy Email address: xxx@apache.org Comment: xxx's key You selected this USER-ID: "xxx-yyy (xxx's key) " Change (N)ame, (C)omment, (E)mail or (O)kay/(Q)uit? o ``` 其中 Real name 需保持和 id.apache.org 中显示的 id 一致。 Email address 为 apache 的邮箱。 ### 3.3 查看和输出 第一行显示公钥文件名（pubring.gpg），第二行显示公钥特征（4096位，Hash字符串和生成时间），第三行显示"用户ID"，第四行显示私钥特征。 ``` $ gpg --list-keys /home/lide/.gnupg/pubring.gpg ----------------------------- pub 4096R/33DBF2E0 2018-12-06 uid xxx-yyy (xxx's key) sub 4096R/0E8182E6 2018-12-06 ``` 其中 xxx-yyy 就是用户ID。 gpg --armor --output public-key.txt --export [用户ID] ``` $ gpg --armor --output public-key.txt --export xxx-yyy $ cat public-key.txt -----BEGIN PGP PUBLIC KEY BLOCK----- Version: GnuPG v2.0.22 (GNU/Linux) mQINBFwJEQ0BEACwqLluHfjBqD/RWZ4uoYxNYHlIzZvbvxAlwS2mn53BirLIU/G3 9opMWNplvmK+3+gNlRlFpiZ7EvHsF/YJOAP59HmI2Z... ``` ## 4. 上传签名公钥 公钥服务器是网络上专门储存用户公钥的服务器。send-keys 参数可以将公钥上传到服务器。 gpg --send-keys xxxx 其中 xxxx 为上一步 --list-keys 结果中 pub 后面的字符串，如上为：33DBF2E0 也可以通过下面的网址上传上述 public-key.txt 的内容： ``` http://keys.gnupg.net ``` 上传成功之后，可以通过查询这个网站，输入 0x33DBF2E0 查询： http://keys.gnupg.net 该网站查询有延迟，可能需要等1个小时。 ## 5. 生成 fingerprint 并上传到 apache 用户信息中 由于公钥服务器没有检查机制，任何人都可以用你的名义上传公钥，所以没有办法保证服务器上的公钥的可靠性。通常，你可以在网站上公布一个公钥指纹，让其他人核对下载到的公钥是否为真。 fingerprint参数生成公钥指纹： ``` gpg --fingerprint [用户ID] ``` ``` $ gpg --fingerprint xxx-yyy pub 4096R/33DBF2E0 2018-12-06 Key fingerprint = 07AA E690 B01D 1A4B 469B 0BEF 5E29 CE39 33DB F2E0 uid xxx-yyy (xxx's key) sub 4096R/0E8182E6 2018-12-06 ``` 将上面的 fingerprint （即 07AA E690 B01D 1A4B 469B 0BEF 5E29 CE39 33DB F2E0）粘贴到自己的用户信息中： https://id.apache.org OpenPGP Public Key Primary Fingerprint: ## 6. 生成 keys 新建一个名为 KEYS 的文件，写入如下内容（无需做任何修改）： ``` This file contains the PGP keys of various developers. Users: pgp < KEYS or gpg --import KEYS Developers: pgp -kxa and append it to this file. or (pgpk -ll && pgpk -xa ) >> this file. or (gpg --list-sigs && gpg --armor --export ) >> this file. ``` 然后生成将 签名信息追加写入： ``` gpg --list-sigs [用户 ID] >> KEYS ``` 最后，将 public key 追加导入： ``` gpg --armor --export [用户 ID] >> KEYS ``` ## 7. 打包签名 如下步骤，也需要通过 SecureCRT 等终端直接登录用户账户，不能通过 su - user 或者 ssh 转，否则密码输入 box 会显示不出来而报错。 ``` $ git checkout 0.9.0-rc01 $ git archive --format=tar 0.9.0-rc01 --prefix=apache-doris-0.9.0-incubating-src/ | gzip > apache-doris-0.9.0-incubating-src.tar.gz $ gpg -u xxx@apache.org --armor --output apache-doris-0.9.0-incubating-src.tar.gz.asc --detach-sign apache-doris-0.9.0-incubating-src.tar.gz $ gpg --verify apache-doris-0.9.0-incubating-src.tar.gz.asc apache-doris-0.9.0-incubating-src.tar.gz $ sha512sum apache-doris-0.9.0-incubating-src.tar.gz > apache-doris-0.9.0-incubating-src.tar.gz.sha512 $ sha512sum --check apache-doris-0.9.0-incubating-src.tar.gz.sha512 ``` ## 8. 上传签名的软件包和 KEYS 文件到 DEV svn 首先，下载 svn 库： ``` svn co https://dist.apache.org/repos/dist/dev/incubator/doris/ ``` 将之前得到的全部文件组织成以下svn路径 ``` ./doris/ ├── 0.9 │   └── 0.9.0-rc1 │   ├── apache-doris-0.9.0-incubating-src.tar.gz │   ├── apache-doris-0.9.0-incubating-src.tar.gz.asc │   ├── apache-doris-0.9.0-incubating-src.tar.gz.sha512 │   └── KEYS ``` 上传这些文件 ``` svn add 0.9.0-rc1 svn commit -m "Release Apache Doris (incubating) 0.9.0 rc1" ``` ## 9. 发社区投票邮件 [VOTE] Release Apache Doris 0.9.0-incubating-rc01 ``` Hi all, Please review and vote on Apache Doris 0.9.0-incubating-rc01 release. The release candidate has been tagged in GitHub as 0.9.0-rc01, available here: https://github.com/apache/incubator-doris/releases/tag/0.9.0-rc01 ===== CHANGE LOG ===== New Features: .... ====================== Thanks to everyone who has contributed to this release. The artifacts (source, signature and checksum) corresponding to this release candidate can be found here: https://dist.apache.org/repos/dist/dev/incubator/doris/0.9/0.9.0-rc1/ This has been signed with PGP key 33DBF2E0, corresponding to lide@apache.org. KEYS file is available here: https://dist.apache.org/repos/dist/dev/incubator/doris/KEYS It is also listed here: https://people.apache.org/keys/committer/lide.asc To verify and build, you can refer to following wiki: https://github.com/apache/incubator-doris/wiki/How-to-verify-Apache-Release https://wiki.apache.org/incubator/IncubatorReleaseChecklist The vote will be open for at least 72 hours. [ ] +1 Approve the release [ ] +0 No opinion [ ] -1 Do not release this package because ... Best Regards, xxx ``` ## 10. 投票通过后，发 Result 邮件 [Result][VOTE] Release Apache Doris 0.9.0-incubating-rc01 ``` Thanks to everyone, and this vote is now closed. It has passed with 4 +1 (binding) votes and no 0 or -1 votes. Binding: +1 Zhao Chun +1 xxx +1 Li Chaoyong +1 Mingyu Chen Best Regards, xxx ``` ## 11. 发邮件到 general@incubator.apache.org 进行投票 [VOTE] Release Apache Doris 0.9.0-incubating-rc01 ``` Hi all, Please review and vote on Apache Doris 0.9.0-incubating-rc01 release. Apache Doris is an MPP-based interactive SQL data warehousing for reporting and analysis. The Apache Doris community has voted on and approved this release: https://lists.apache.org/thread.html/d70f7c8a8ae448bf6680a15914646005c6483564464cfa15f4ddc2fc@%3Cdev.doris.apache.org%3E The vote result email thread: https://lists.apache.org/thread.html/64d229f0ba15d66adc83306bc8d7b7ccd5910ecb7e842718ce6a61da@%3Cdev.doris.apache.org%3E The release candidate has been tagged in GitHub as 0.9.0-rc01, available here: https://github.com/apache/incubator-doris/releases/tag/0.9.0-rc01 There is no CHANGE LOG file because this is the first release of Apache Doris. Thanks to everyone who has contributed to this release, and there is a simple release notes can be found here: https://github.com/apache/incubator-doris/issues/406 The artifacts (source, signature and checksum) corresponding to this release candidate can be found here: https://dist.apache.org/repos/dist/dev/incubator/doris/0.9/0.9.0-rc01/ This has been signed with PGP key 33DBF2E0, corresponding to lide@apache.org. KEYS file is available here: https://dist.apache.org/repos/dist/dev/incubator/doris/KEYS It is also listed here: https://people.apache.org/keys/committer/lide.asc The vote will be open for at least 72 hours. [ ] +1 Approve the release [ ] +0 No opinion [ ] -1 Do not release this package because ... To verify and build, you can refer to following instruction: Firstly, you must be install and start docker service, and then you could build Doris as following steps: Step1: Pull the docker image with Doris building environment $ docker pull apachedoris/doris-dev:build-env You can check it by listing images, its size is about 3.28GB. Step2: Run the Docker image You can run image directly: $ docker run -it apachedoris/doris-dev:build-env Step3: Download Doris source Now you should in docker environment, and you can download Doris source package. (If you have downloaded source and it is not in image, you can map its path to image in Step2.) $ wget https://dist.apache.org/repos/dist/dev/incubator/doris/0.9/0.9.0-rc01/apache-doris-0.9.0.rc01-incubating-src.tar.gz Step4: Build Doris Now you can decompress and enter Doris source path and build Doris. $ tar zxvf apache-doris-0.9.0.rc01-incubating-src.tar.gz $ cd apache-doris-0.9.0.rc01-incubating-src $ sh build.sh Best Regards, xxx ``` 邮件的 thread 连接可以在这里找到： `https://lists.apache.org/list.html?dev@doris.apache.org` ## 12. 发 Result 邮件到 general@incubator.apache.org [RESULT][VOTE] Release Apache Doris 0.9.0-incubating-rc01 ``` Hi, Thanks to everyone, and the vote for releasing Apache Doris 0.9.0-incubating-rc01 is now closed. It has passed with 4 +1 (binding) votes and no 0 or -1 votes. Binding: +1 Willem Jiang +1 Justin Mclean +1 ShaoFeng Shi +1 Makoto Yui The vote thread: https://lists.apache.org/thread.html/da05fdd8d84e35de527f27200b5690d7811a1e97d419d1ea66562130@%3Cgeneral.incubator.apache.org%3E Best Regards, xxx ``` ## 13. 上传 package 到 release 当正式发布投票成功后，先发[Result]邮件，然后就准备 release package。 将之前在dev下发布的对应rc文件夹下的源码包、签名文件和hash文件拷贝到另一个目录 0.9.0-incubating，注意文件名字中不要rcxx (可以rename，但不要重新计算签名，hash可以重新计算，结果不会变) 第一次发布的话 KEYS 文件也需要拷贝过来。然后add到svn release 下。 ``` https://dist.apache.org/repos/dist/release/incubator/doris/0.9.0-incubating/ 最终能在 apache 官网看到： http://www.apache.org/dist/incubator/doris/0.9.0-incubating/ ``` ## 14. 发 Announce 邮件到 general@incubator.apache.org Title: ``` [ANNOUNCE] Apache Doris (incubating) 0.9.0 Release ``` 发送邮件组： ``` general@incubator.apache.org dev@doris.apache.org ``` 邮件正文： ``` Hi All, We are pleased to announce the release of Apache Doris 0.9.0-incubating. Apache Doris (incubating) is an MPP-based interactive SQL data warehousing for reporting and analysis. The release is available at: http://doris.apache.org/downloads.html Thanks to everyone who has contributed to this release, and the release note can be found here: https://github.com/apache/incubator-doris/releases Best Regards, On behalf of the Doris team, xxx ``` ## 15. 在 Doris 官网和 github 发布链接 ### 15.1 创建下载链接 下载链接： http://www.apache.org/dyn/closer.cgi?filename=incubator/doris/0.9.0-incubating/apache-doris-0.9.0-incubating-src.tar.gz&action=download wget --trust-server-names "https://www.apache.org/dyn/mirrors/mirrors.cgi?action=download&filename=incubator/doris/0.9.0-incubating/apache-doris-0.9.0-incubating-src.tar.gz" 原始位置: https://www.apache.org/dist/incubator/doris/0.9.0-incubating/ http://www.apache.org/dyn/closer.cgi/incubator/doris/0.9.0-incubating/apache-doris-0.9.0-incubating-src.tar.gz 源码包（source package）: http://www.apache.org/dyn/closer.cgi/incubator/doris/0.9.0-incubating/apache-doris-0.9.0-incubating-src.tar.gz ASC: http://archive.apache.org/dist/incubator/doris/0.9.0-incubating/apache-doris-0.9.0-incubating-src.tar.gz.asc sha512: http://archive.apache.org/dist/incubator/doris/0.9.0-incubating/apache-doris-0.9.0-incubating-src.tar.gz.sha512 KEYS: http://archive.apache.org/dist/incubator/doris/KEYS refer to: ### 15.2 准备 release note 需要修改如下两个地方： 1、Github 的 release 页面 ``` https://github.com/apache/incubator-doris/releases/tag/0.9.0-rc01 ``` 2、Doris 官网下载页面 ``` http://doris.apache.org/downloads.html ```