Commit Graph

17 Commits

Author SHA1 Message Date
911e7cc360 *: enable revive (#35970)
ref pingcap/tidb#35345
2022-07-15 16:03:06 +08:00
7647c2b9b0 parser: parse gbk encoding SQL token by token instead of the whole SQL (#31039)
close pingcap/tidb#30765
2021-12-28 22:07:49 +08:00
xhe
94e30df8e2 parser: fix compiling, add unit tests (#28700) 2021-10-11 23:15:25 +08:00
b266e09b30 [parser] charset: clean up some code about charset (#1306) 2021-10-09 14:53:23 +08:00
6b60fb9717 [parser] *: Add Digest struct to get bytes of digest (#1231) 2021-10-09 14:53:23 +08:00
xhe
bc860cdd5e [parser] *: fix static check report (#1195)
* ast: base.go

Signed-off-by: xhe <xw897002528@gmail.com>

* ast: expressions.go

Signed-off-by: xhe <xw897002528@gmail.com>

* ast: misc_test.go

Signed-off-by: xhe <xw897002528@gmail.com>

* *: digester.go

Signed-off-by: xhe <xw897002528@gmail.com>

* goyacc: format_yacc.go

Signed-off-by: xhe <xw897002528@gmail.com>

* *: lexer_test.go

Signed-off-by: xhe <xw897002528@gmail.com>

* *: misc.go

Signed-off-by: xhe <xw897002528@gmail.com>

* model: model.go

Signed-off-by: xhe <xw897002528@gmail.com>

* model: model_test.go

Signed-off-by: xhe <xw897002528@gmail.com>

* *: parser_test.go

Signed-off-by: xhe <xw897002528@gmail.com>

* test_driver: test_driver_datum.go

Signed-off-by: xhe <xw897002528@gmail.com>

* test_driver: test_driver_mydecimal.go

Signed-off-by: xhe <xw897002528@gmail.com>

* ast: ddl.go, bugfix

Signed-off-by: xhe <xw897002528@gmail.com>

* mysql: errcode.go

charset/charset.go is passing errcode to NewStd, which requires int. But
here codes are uint16. To get rid of the error, uint16 is removed.

Signed-off-by: xhe <xw897002528@gmail.com>

* model: better test

Signed-off-by: xhe <xw897002528@gmail.com>

* *: fix timezone

Signed-off-by: xhe <xw897002528@gmail.com>

Co-authored-by: ti-srebot <66930949+ti-srebot@users.noreply.github.com>
2021-10-09 14:53:23 +08:00
7515d165a3 [parser] parser: regard NULL as literal when normalizing query (#1166)
* parser: regard NULL as literal when normalizing queries

Currently, Normalize() converts "SELECT NULL" to "select null", while NULL is
regarded as a literal in MySQL, as described in the documentation.
https://dev.mysql.com/doc/refman/8.0/en/literals.html
It seems natural that Normalize() would convert "SELECT NULL" to "select ?"
from the consistency viewpoint.

* Handle case of 'select `null`'

* Fix quotation

Co-authored-by: Arenatlx <ailinsilence4@gmail.com>
Co-authored-by: kennytm <kennytm@gmail.com>
2021-10-09 14:53:23 +08:00
b3efddbbec [parser] digest: add missed backquote when getting digest (#1176)
* fix test

* add test
2021-10-09 14:53:23 +08:00
9c4959a061 [parser] lexer: quote identifier when get sql digest (#1151)
* fix

* fix

* Update digester.go

Co-authored-by: kennytm <kennytm@gmail.com>

Co-authored-by: kennytm <kennytm@gmail.com>
2021-10-09 14:53:23 +08:00
5d55a9dd61 [parser] digester: keep the singleAtIdentifier in normalize (#1128) 2021-10-09 14:53:23 +08:00
8d06dff8f2 [parser] digester: consider paramMarker as literal when normalizing SQL (#1040)
* digester: consider paramMarker when normalizing SQL

* add tests

* delete unnecessary changes

Co-authored-by: Kenan Yao <cauchy1992@gmail.com>
2021-10-09 14:53:23 +08:00
6eb2909901 [parser] change all dot import, according to staticcheck (#730)
* change all dot import, according to staticcheck

* make staticcheck happy
2021-10-09 14:53:23 +08:00
5cfb690491 [parser] Rewrite special comment parser (#711)
* lexer: added a function to scan version digits

* lexer: removed special comment scanner

Instead, simply strip off '/*!' and the pairing '*/'.

Temporarily disabled handling of /*+ ... */

* lexer: support collecting the entire /*+ ... */ as a single token

This new token has type `hintComment`. The actual hint will be parsed
lazily.

* lexer,yy_parser: move lastErrorAsWarn() from Parser into Scanner

* goyacc: do not allow conflict, support changing parser type name

change the "DO NOT EDIT" line to fit the Go standard

* parser,hintparser: created a new parser just for parsing optimizer hints

deleted all optimizer hint rules from parser.go

refactor the Makefile to support building both parsers (also deleted some
outdated fixup of *parser.go)

* lexer: fix comment parser

* codecov: don't wait for integration test before showing the coverage

* lexer: fix comment parsing again

* hintparser: TiDB still expects `HASH_JOIN(@qb1 foo@qb2)` to be valid :(

* parser,hintparser: provide the true line/column offset in the warnings

cache the hintparser in the main parser to avoid repeated allocation

* lexer: delete unused sqlOffsetInComment()
2021-10-09 14:53:23 +08:00
a7ac29bd99 [parser] parser: introduce DigestNormalized and deprecate DigestHash (#656) 2021-10-09 14:53:23 +08:00
824bc91f46 [parser] digester: avoid infinite loop when an invalid token is seen (#313) 2021-10-09 14:53:23 +08:00
f6cb990bb0 [parser] normalize and generate digest at same time (#215) 2021-10-09 14:53:23 +08:00
fee9359019 [parser] support sql digest (#32) 2021-10-09 14:53:23 +08:00