database/doris
2
0
Fork 0
Code Issues Pull Requests Actions 3 Packages Projects Releases Wiki Activity
Files
2c24fe80fa76d4afbeeecaacbb11e29e07e84fd9
doris/tools/show_segment_status
History
Mingyu Chen 67b842ce04 [License] Organize and modify the license of the code (#4371) 
1. Disable the MySQL client and LZO library by default when building the Doris.

    MySQL client library is used for MySQL external table feature.
    This feature will be replaced by the new ODBC external table soon.

    LZO library is used to compress/decompress data of some old data format of Doris,
    which is no longer used anymore.

2. Add missing license to some files.

3. For all non-Apache-License code, all are explained in NOTICE file and the corresponding license is declared.

4. Remove the js source code from webroot, it will be downloaded as thirdparty
2020-08-24 21:51:55 +08:00
..
be_tablet_reslover.py
Add a tool to show segment status (#2260)
2019-11-27 11:35:16 +08:00
conf
[License] Organize and modify the license of the code (#4371)
2020-08-24 21:51:55 +08:00
fe_meta_resolver.py
Add a tool to show segment status (#2260)
2019-11-27 11:35:16 +08:00
README.md
[SegmentV2] Optimize the upgrade logic of SegmentV2 (#3340)
2020-04-21 10:45:29 +08:00
show_segment_status.py
Add a tool to show segment status (#2260)
2019-11-27 11:35:16 +08:00

README.md

This tool is used to get the progress of all current table transitions during the online segment_2 function.

Currently, you can specify 3 dimensions (in the conf file) to view the results, you can specify one of them individually, or you can customize the combination (that is, specify multiple at the same time).

Note

We use MySQLdb python lib to fetch meta from FE, so you must install it.

You can get MySQLdb lib from https://pypi.python.org/pypi/MySQL-python, then you can install it as follows:

$ tar zxvf MySQL-python-*.tar.gz
$ cd MySQL-python-*
$ python setup.py build
$ python setup.py install

Steps

  1. Fill in the conf according to your cluster configuration, and specify the table or be you want to watch.
  2. Execute python show_segment_status.py

Example

  1. If you want to watch the process of a table named xxxx, you can specify table_name = xxxx in conf file

  2. If you want to watch the process on be whose be_id is xxxx, you can specify be_id = xxxx in conf file

Output Example Format

The first number is about segment v2, the latter one is the total count.

==========SUMMARY()===========
rowset_count: 289845 / 289845
rowset_disk_size: 84627551189 / 84627551189
rowset_row_count: 1150899153 / 1150899153
===========================================================
==========SUMMARY(table=xxxx)===========
rowset_count: 289845 / 289845
rowset_disk_size: 84627551189 / 84627551189
rowset_row_count: 1150899153 / 1150899153
===========================================================
==========SUMMARY(be=10003 )===========
rowset_count: 79650 / 79650
rowset_disk_size: 24473921575 / 24473921575
rowset_row_count: 331449328 / 331449328
===========================================================