!23 rework: 解决uint数据类型查询报错

Merge pull request !23 from zcl/rework#bug#1

README.md

@@ -1,68 +1,104 @@
 # openGauss-connector-python-psycopg2
 
-#### 介绍
+## 介绍
 
 该仓库为openGauss的python驱动。
 
-#### 软件架构
-
-
-#### 打包步骤
+## 打包步骤
 
 1. 推荐使用python3，请在服务器上装python3和python3-devel
-```
-yum install python3 python3-devel
-```
 
-2. python驱动的打包需要依赖openGauss-server编译的libpq，请参照 [openGauss使用命令编译代码](https://gitee.com/opengauss/openGauss-server#%E4%BD%BF%E7%94%A8%E5%91%BD%E4%BB%A4%E7%BC%96%E8%AF%91%E4%BB%A3%E7%A0%81) 
+    ```shell
+    yum install python3 python3-devel
+    ```
 
-3. 下载`openGauss-connector-python-psycopg2`仓库代码，进入到根目录下，执行
-```
-sh build.sh -bd /data/compile/openGauss-server/dest/ -v 5.0.0
-```
-**说明** \
--bd: 指定openGauss数据库构建结果目录 \
--v: 指定构建包的版本号。不指定则默认为5.0.0
+2. python驱动的打包需要依赖openGauss-server编译的libpq，请参照 [openGauss使用命令编译代码](https://gitee.com/opengauss/openGauss-server#%E4%BD%BF%E7%94%A8%E5%91%BD%E4%BB%A4%E7%BC%96%E8%AF%91%E4%BB%A3%E7%A0%81)
 
-编译完成后的驱动，在 `output` 目录下。
+3. 下载 `openGauss-connector-python-psycopg2` 仓库代码，进入到根目录下，执行
 
-#### 软件下载
+    ```shell
+    sh build.sh -bd /data/compile/openGauss-server/dest/ -v 5.0.0
+    ```
+
+    **说明** \
+    -bd: 指定openGauss数据库构建结果目录 \
+    -v: 指定构建包的版本号。不指定则默认为5.0.0
+
+    编译完成后的驱动，在 `output` 目录下。
+
+## 软件下载
 
 社区每日构建，提供三套环境已经编译好的驱动包供使用，下载路径如下：
 
-其中 **VERSION** 为当前最新的版本号。
+其中 **\<VERSION\>** 为当前最新的版本号。
 
-openEuler-aarch64：\
-https://opengauss.obs.cn-south-1.myhuaweicloud.com/latest/arm/openGauss-**VERSION**-openEuler-aarch64-Python.tar.gz
+openEuler-aarch64:
 
+`https://opengauss.obs.cn-south-1.myhuaweicloud.com/latest/arm/openGauss-<VERSION>-openEuler-aarch64-Python.tar.gz`
 
-CentOS-x86_64: \
-https://opengauss.obs.cn-south-1.myhuaweicloud.com/latest/x86/openGauss-**VERSION**-CentOS-x86_64-Python.tar.gz
+CentOS-x86_64:
 
+`https://opengauss.obs.cn-south-1.myhuaweicloud.com/latest/x86/openGauss-<VERSION>-CentOS-x86_64-Python.tar.gz`
 
-openEuler-x86_64: \
-https://opengauss.obs.cn-south-1.myhuaweicloud.com/latest/x86_openEuler/openGauss-**VERSION**-openEuler-x86_64-Python.tar.gz
+openEuler-x86_64:
 
+`https://opengauss.obs.cn-south-1.myhuaweicloud.com/latest/x86_openEuler/openGauss-<VERSION>-openEuler-x86_64-Python.tar.gz`
 
-#### 使用说明
+## 使用说明
 
-1.  xxxx
-2.  xxxx
-3.  xxxx
+### 安装 psycopg2
 
-#### 参与贡献
+解压安装包后，会得到两个目录 `lib` 和 `psycopg2`。
 
-1.  Fork 本仓库
-2.  新建 Feat_xxx 分支
-3.  提交代码
-4.  新建 Pull Request
+接下来您需要将 psycopg2 目录拷贝到 Python 解释器的 site-packages 下，让 python 可以索引到
 
+可以通过如下指令来查找 site-packages 目录所在位置:
 
-#### 特技
+```shell
+python -c "from distutils.sysconfig import get_python_lib;print(get_python_lib())"
+# output: /usr/lib/python3.6/site-packages
+```
 
-1.  使用 Readme\_XXX.md 来支持不同的语言，例如 Readme\_en.md, Readme\_zh.md
-2.  Gitee 官方博客 [blog.gitee.com](https://blog.gitee.com)
-3.  你可以 [https://gitee.com/explore](https://gitee.com/explore) 这个地址来了解 Gitee 上的优秀开源项目
-4.  [GVP](https://gitee.com/gvp) 全称是 Gitee 最有价值开源项目，是综合评定出的优秀开源项目
-5.  Gitee 官方提供的使用手册 [https://gitee.com/help](https://gitee.com/help)
-6.  Gitee 封面人物是一档用来展示 Gitee 会员风采的栏目 [https://gitee.com/gitee-stars/](https://gitee.com/gitee-stars/)
+确定 site-packages 的目录位置后，将 psycopg2 文件夹整个拷贝该目录下，并授权便于其他用户使用
+
+```shell
+cp -r psycopg2 [/path/to/site-packages/] && chmod 755 [/path/to/site-packages/]psycopg2
+```
+
+安装包中的另一个目录 `lib` 是 psycopg2 依赖的 libpq 等 C 动态库文件。
+
+这个目录中的文件可能会和系统的动态库存在重名冲突的问题，
+
+所以建议将应用程序和 lib 目录存放到单独的一个系统用户下和系统自带库进行隔离，避免影响系统其他工具。
+
+切换到运行用户下，执行如下命令进行设置：
+
+```shell
+echo "export LD_LIBRARY_PATH=[/path/to/lib]:$LD_LIBRARY_PATH" >> ~/.bashrc
+source ~/.bashrc
+```
+
+这样就安装完成了。
+
+### 应用连接
+
+```python
+import psycopg2
+
+# dsn 格式
+conn = psycopg2.connect("postgres://user:password@ip1:port,ip2:port:.../dbname?target_session_attrs=[any|read-write]")
+# key-value 格式
+conn = psycopg2.connect(host="ip1[,ip2]",
+                        port=port1,
+                        database=db,
+                        user=user,
+                        password=password,
+                        target_session_attrs="read-write")
+```
+
+## 参与贡献
+
+1. Fork 本仓库
+2. 新建 Feat_xxx 分支
+3. 提交代码
+4. 新建 Pull Request

lib/extras.py

@@ -140,15 +140,15 @@ class DictCursor(DictCursorBase):
         super().__init__(*args, **kwargs)
         self._prefetch = True
 
-    def execute(self, query, vars=None):
+    def execute(self, query, vars=None, place_holder = '%'):
         self.index = OrderedDict()
         self._query_executed = True
-        return super().execute(query, vars)
+        return super().execute(query, vars, place_holder)
 
-    def callproc(self, procname, vars=None):
+    def callproc(self, procname, vars=None, place_holder = '%'):
         self.index = OrderedDict()
         self._query_executed = True
-        return super().callproc(procname, vars)
+        return super().callproc(procname, vars, place_holder)
 
     def _build_index(self):
         if self._query_executed and self.description:
@@ -230,15 +230,15 @@ class RealDictCursor(DictCursorBase):
         kwargs['row_factory'] = RealDictRow
         super().__init__(*args, **kwargs)
 
-    def execute(self, query, vars=None):
+    def execute(self, query, vars=None, place_holder = '%'):
         self.column_mapping = []
         self._query_executed = True
-        return super().execute(query, vars)
+        return super().execute(query, vars, place_holder)
 
-    def callproc(self, procname, vars=None):
+    def callproc(self, procname, vars=None, place_holder = '%'):
         self.column_mapping = []
         self._query_executed = True
-        return super().callproc(procname, vars)
+        return super().callproc(procname, vars, place_holder)
 
     def _build_index(self):
         if self._query_executed and self.description:
@@ -307,17 +307,17 @@ class NamedTupleCursor(_cursor):
     Record = None
     MAX_CACHE = 1024
 
-    def execute(self, query, vars=None):
+    def execute(self, query, vars=None, place_holder = '%'):
         self.Record = None
-        return super().execute(query, vars)
+        return super().execute(query, vars, place_holder)
 
-    def executemany(self, query, vars):
+    def executemany(self, query, vars, place_holder = '%'):
         self.Record = None
-        return super().executemany(query, vars)
+        return super().executemany(query, vars, place_holder)
 
-    def callproc(self, procname, vars=None):
+    def callproc(self, procname, vars=None, place_holder = '%'):
         self.Record = None
-        return super().callproc(procname, vars)
+        return super().callproc(procname, vars, place_holder)
 
     def fetchone(self):
         t = super().fetchone()
@@ -444,15 +444,15 @@ class LoggingConnection(_connection):
 class LoggingCursor(_cursor):
     """A cursor that logs queries using its connection logging facilities."""
 
-    def execute(self, query, vars=None):
+    def execute(self, query, vars=None, place_holder = '%'):
         try:
-            return super().execute(query, vars)
+            return super().execute(query, vars, place_holder)
         finally:
             self.connection.log(self.query, self)
 
-    def callproc(self, procname, vars=None):
+    def callproc(self, procname, vars=None, place_holder = '%'):
         try:
-            return super().callproc(procname, vars)
+            return super().callproc(procname, vars, place_holder)
         finally:
             self.connection.log(self.query, self)
 
@@ -488,13 +488,13 @@ class MinTimeLoggingConnection(LoggingConnection):
 class MinTimeLoggingCursor(LoggingCursor):
     """The cursor sub-class companion to `MinTimeLoggingConnection`."""
 
-    def execute(self, query, vars=None):
+    def execute(self, query, vars=None, place_holder = '%'):
         self.timestamp = _time.time()
-        return LoggingCursor.execute(self, query, vars)
+        return LoggingCursor.execute(self, query, vars, place_holder)
 
-    def callproc(self, procname, vars=None):
+    def callproc(self, procname, vars=None, place_holder = '%'):
         self.timestamp = _time.time()
-        return LoggingCursor.callproc(self, procname, vars)
+        return LoggingCursor.callproc(self, procname, vars, place_holder)
 
 
 class LogicalReplicationConnection(_replicationConnection):
@@ -1099,8 +1099,7 @@ ORDER BY attnum;
         recs = curs.fetchall()
 
         # revert the status of the connection as before the command
-        if (conn_status != _ext.STATUS_IN_TRANSACTION
-        and not conn.autocommit):
+        if conn_status != _ext.STATUS_IN_TRANSACTION and not conn.autocommit:
             conn.rollback()
 
         if not recs:
@@ -1143,7 +1142,7 @@ def register_composite(name, conn_or_curs, globally=False, factory=None):
     return caster
 
 
-def _paginate(seq, page_size):
+def _paginate(seq, page_size, to_byte=False):
     """Consume an iterable and return it in chunks.
 
     Every chunk is at most `page_size`. Never return an empty chunk.
@@ -1153,7 +1152,16 @@ def _paginate(seq, page_size):
     while True:
         try:
             for i in range(page_size):
-                page.append(next(it))
+                if not to_byte:
+                    page.append(next(it))
+                    continue
+                vs = next(it)
+                if isinstance(vs, (list, tuple)):
+                    # Ignore None object
+                    # Serialized params to bytes
+                    page.append(list(map(lambda v: v if v is None else str(v).encode('utf-8'), vs)))
+                else:
+                    page.append(vs)
             yield page
             page = []
         except StopIteration:
@@ -1161,8 +1169,7 @@ def _paginate(seq, page_size):
                 yield page
             return
 
-
-def execute_batch(cur, sql, argslist, page_size=100):
+def execute_batch(cur, sql, argslist, page_size=100, place_holder = '%'):
     r"""Execute groups of statements in fewer server roundtrips.
 
     Execute *sql* several times, against all parameters set (sequences or
@@ -1183,11 +1190,11 @@ def execute_batch(cur, sql, argslist, page_size=100):
 
     """
     for page in _paginate(argslist, page_size=page_size):
-        sqls = [cur.mogrify(sql, args) for args in page]
+        sqls = [cur.mogrify(sql, args, place_holder) for args in page]
         cur.execute(b";".join(sqls))
 
 
-def execute_values(cur, sql, argslist, template=None, page_size=100, fetch=False):
+def execute_values(cur, sql, argslist, template=None, page_size=100, fetch=False, place_holder = '%'):
     '''Execute a statement using :sql:`VALUES` with a sequence of parameters.
 
     :param cur: the cursor to use to execute the query.
@@ -1264,7 +1271,7 @@ def execute_values(cur, sql, argslist, template=None, page_size=100, fetch=False
             template = b'(' + b','.join([b'%s'] * len(page[0])) + b')'
         parts = pre[:]
         for args in page:
-            parts.append(cur.mogrify(template, args))
+            parts.append(cur.mogrify(template, args, place_holder))
             parts.append(b',')
         parts[-1:] = post
         cur.execute(b''.join(parts))
@@ -1304,3 +1311,39 @@ def _split_sql(sql):
         raise ValueError("the query doesn't contain any '%s' placeholder")
 
     return pre, post
+
+
+def execute_prepared_batch(cur, prepared_statement_name, args_list, page_size=100):
+    r"""
+    [openGauss libpq only]
+
+    Execute prepared statement with api `PQexecPreparedBatch` (new api in openGauss's libpq.so)
+
+    Arguments:
+        argslist: Two-dimensional list, if empty, return directly
+        Each parameter in the argument list must be a string or be string-able(should implements `__str__` magic method)
+    """
+    if len(args_list) == 0:
+        return
+
+    nparams = len(args_list[0])
+    for page in _paginate(args_list, page_size=page_size, to_byte=True):
+            cur.execute_prepared_batch(prepared_statement_name, nparams, len(page), page)
+
+
+def execute_params_batch(cur, sql_format, args_list, page_size=100):
+    r"""
+    [openGauss libpq only]
+
+    Execute sql with api `PQexecParamsBatch` (new api in openGauss's libpq.so)
+
+    Arguments:
+        argslist: Two-dimensional list, if empty, return directly
+        Each parameter in the argument list must be a string or be string-able(should implements `__str__` magic method)
+    """
+    if len(args_list) == 0:
+        return
+
+    nparams = len(args_list[0])
+    for page in _paginate(args_list, page_size=page_size, to_byte=True):
+        cur.execute_params_batch(sql_format, nparams, len(page), page)

psycopg/bytes_format.c

@@ -78,6 +78,8 @@
  * Agreement.
  */
 
+#include <string.h>
+
 #define PSYCOPG_MODULE
 #include "psycopg/psycopg.h"
 #include "pyport.h"
@@ -85,225 +87,325 @@
 /* Helpers for formatstring */
 
 BORROWED Py_LOCAL_INLINE(PyObject *)
-getnextarg(PyObject *args, Py_ssize_t arglen, Py_ssize_t *p_argidx)
-{
+getnextarg(PyObject *args, Py_ssize_t arglen, Py_ssize_t *p_argidx) {
     Py_ssize_t argidx = *p_argidx;
     if (argidx < arglen) {
         (*p_argidx)++;
-        if (arglen < 0)
-            return args;
-        else
-            return PyTuple_GetItem(args, argidx);
+        if (arglen < 0) return args;
+        else return PyTuple_GetItem(args, argidx);
     }
-    PyErr_SetString(PyExc_TypeError,
-                    "not enough arguments for format string");
     return NULL;
 }
 
+/*
+    for function getnextarg：
+    I delete the line including 'raise error', for making this func a iterator
+    just used to fill in the arguments array
+*/
+
 /* wrapper around _Bytes_Resize offering normal Python call semantics */
 
 STEALS(1)
 Py_LOCAL_INLINE(PyObject *)
 resize_bytes(PyObject *b, Py_ssize_t newsize) {
-    if (0 == _Bytes_Resize(&b, newsize)) {
-        return b;
-    }
-    else {
-        return NULL;
-    }
+    if (0 == _Bytes_Resize(&b, newsize)) return b;
+    else return NULL;
 }
 
-/* fmt%(v1,v2,...) is roughly equivalent to sprintf(fmt, v1, v2, ...) */
-
-PyObject *
-Bytes_Format(PyObject *format, PyObject *args)
-{
-    char *fmt, *res;
-    Py_ssize_t arglen, argidx;
-    Py_ssize_t reslen, rescnt, fmtcnt;
-    int args_owned = 0;
-    PyObject *result;
-    PyObject *dict = NULL;
-    if (format == NULL || !Bytes_Check(format) || args == NULL) {
+PyObject *Bytes_Format(PyObject *format, PyObject *args, char place_holder) {
+    char *fmt, *res;                    // array pointer of format, and array pointer of result
+    Py_ssize_t arglen, argidx;          // length of arguments array, and index of arguments(when processing args_list)
+    Py_ssize_t reslen, rescnt, fmtcnt;  // rescnt: blank space in result; reslen: the total length of result; fmtcnt: length of format
+    int args_owned = 0;                 // args is valid or invalid(or maybe refcnt), 0 for invalid，1 otherwise
+    PyObject *result;                   // function's return value
+    PyObject *dict = NULL;              // dictionary
+    PyObject *args_value = NULL;        // every argument store in it after parse
+    char **args_list = NULL;            // arguments list as char **
+    char *args_buffer = NULL;           // Bytes_AS_STRING(args_value)
+    Py_ssize_t * args_len = NULL;       // every argument's length in args_list
+    int args_id = 0;                    // index of arguments(when generating result and check args of no use)
+    int index_type = 0;                 // if exists $number, it will be 1, otherwise 0
+    int *arg_usecnt = NULL;             // used to test if args are all used
+    
+    if (format == NULL || !Bytes_Check(format) || args == NULL) {   // check if arguments are valid
         PyErr_SetString(PyExc_SystemError, "bad argument to internal function");
         return NULL;
     }
-    fmt = Bytes_AS_STRING(format);
-    fmtcnt = Bytes_GET_SIZE(format);
-    reslen = rescnt = fmtcnt + 100;
-    result = Bytes_FromStringAndSize((char *)NULL, reslen);
-    if (result == NULL)
-        return NULL;
+    fmt = Bytes_AS_STRING(format);      // get pointer of format
+    fmtcnt = Bytes_GET_SIZE(format);    // get length of format
+    reslen = rescnt = 1;
+    while (reslen <= fmtcnt) {          // when space is not enough, double it's size
+        reslen *= 2;
+        rescnt *= 2;
+    }
+    result = Bytes_FromStringAndSize((char *)NULL, reslen); 
+    if (result == NULL) return NULL;
     res = Bytes_AS_STRING(result);
-    if (PyTuple_Check(args)) {
+    if (PyTuple_Check(args)) {          // check if arguments are sequences
         arglen = PyTuple_GET_SIZE(args);
         argidx = 0;
     }
-    else {
+    else {                              // if no, then this two are of no importance
         arglen = -1;
         argidx = -2;
     }
-    if (Py_TYPE(args)->tp_as_mapping && !PyTuple_Check(args) &&
-        !PyObject_TypeCheck(args, &Bytes_Type))
+    if (Py_TYPE(args)->tp_as_mapping && !PyTuple_Check(args) && !PyObject_TypeCheck(args, &Bytes_Type)) {   // check if args is dict
         dict = args;
-    while (--fmtcnt >= 0) {
-        if (*fmt != '%') {
+        // Py_INCREF(dict);
+    }
+    while (--fmtcnt >= 0) {             // scan the format
+        if (*fmt != '%') {              // if not %, pass it(for the special format '%(name)s')
             if (--rescnt < 0) {
-                rescnt = fmtcnt + 100;
-                reslen += rescnt;
+                rescnt = reslen;        // double the space
+                reslen *= 2;
                 if (!(result = resize_bytes(result, reslen))) {
                     return NULL;
                 }
-                res = Bytes_AS_STRING(result) + reslen - rescnt;
+                res = Bytes_AS_STRING(result) + reslen - rescnt;// calculate offset
                 --rescnt;
             }
-            *res++ = *fmt++;
+            *res++ = *fmt++;            // copy
         }
         else {
             /* Got a format specifier */
-            Py_ssize_t width = -1;
-            int c = '\0';
-            PyObject *v = NULL;
-            PyObject *temp = NULL;
-            char *pbuf;
-            Py_ssize_t len;
             fmt++;
             if (*fmt == '(') {
-                char *keystart;
-                Py_ssize_t keylen;
+                char *keystart;         // begin pos of left bracket
+                Py_ssize_t keylen;      // length of content in bracket
                 PyObject *key;
-                int pcount = 1;
-
+                int pcount = 1;         // counter of left bracket
+                Py_ssize_t length = 0;
                 if (dict == NULL) {
-                    PyErr_SetString(PyExc_TypeError,
-                             "format requires a mapping");
+                    PyErr_SetString(PyExc_TypeError, "format requires a mapping");
                     goto error;
                 }
                 ++fmt;
                 --fmtcnt;
                 keystart = fmt;
                 /* Skip over balanced parentheses */
-                while (pcount > 0 && --fmtcnt >= 0) {
-                    if (*fmt == ')')
-                        --pcount;
-                    else if (*fmt == '(')
-                        ++pcount;
+                while (pcount > 0 && --fmtcnt >= 0) {       // find the matching right bracket
+                    if (*fmt == ')') --pcount;
+                    else if (*fmt == '(') ++pcount;
                     fmt++;
                 }
                 keylen = fmt - keystart - 1;
-                if (fmtcnt < 0 || pcount > 0) {
-                    PyErr_SetString(PyExc_ValueError,
-                               "incomplete format key");
+                if (fmtcnt < 0 || pcount > 0 || *(fmt++) != 's') {             // not found, raise an error
+                    PyErr_SetString(PyExc_ValueError, "incomplete format key");
                     goto error;
                 }
-                key = Text_FromUTF8AndSize(keystart, keylen);
-                if (key == NULL)
-                    goto error;
-                if (args_owned) {
+                --fmtcnt;
+                key = Text_FromUTF8AndSize(keystart, keylen);// get key
+                if (key == NULL) goto error;
+                if (args_owned) {                           // if refcnt > 0, then release
                     Py_DECREF(args);
                     args_owned = 0;
                 }
-                args = PyObject_GetItem(dict, key);
+                args = PyObject_GetItem(dict, key);         // get value with key
                 Py_DECREF(key);
-                if (args == NULL) {
+                if (args == NULL) goto error;
+                if (!Bytes_CheckExact(args)) {
+                    PyErr_Format(PyExc_ValueError, "only bytes values expected, got %s", Py_TYPE(args)->tp_name);   // raise error, but may have bug
                     goto error;
                 }
+                args_buffer = Bytes_AS_STRING(args);        // temporary buffer
+                length = Bytes_GET_SIZE(args);
+                if (rescnt < length) {
+                    while (rescnt < length) {
+                        rescnt += reslen;
+                        reslen *= 2;
+                    }
+                    if ((result = resize_bytes(result, reslen)) == NULL) goto error;
+                }
+                res = Bytes_AS_STRING(result) + reslen - rescnt;
+                Py_MEMCPY(res, args_buffer, length);
+                rescnt -= length;
+                res += length;
                 args_owned = 1;
-                arglen = -1;
+                arglen = -1;    // exists place holder as "%(name)s", set these arguments to invalid
                 argidx = -2;
             }
-            while (--fmtcnt >= 0) {
-                c = *fmt++;
-                break;
-            }
-            if (fmtcnt < 0) {
-                PyErr_SetString(PyExc_ValueError,
-                                "incomplete format");
-                goto error;
-            }
-            switch (c) {
-            case '%':
-                pbuf = "%";
-                len = 1;
-                break;
-            case 's':
-                /* only bytes! */
-                if (!(v = getnextarg(args, arglen, &argidx)))
-                    goto error;
-                if (!Bytes_CheckExact(v)) {
-                    PyErr_Format(PyExc_ValueError,
-                                    "only bytes values expected, got %s",
-                                    Py_TYPE(v)->tp_name);
-                    goto error;
-                }
-                temp = v;
-                Py_INCREF(v);
-                pbuf = Bytes_AS_STRING(temp);
-                len = Bytes_GET_SIZE(temp);
-                break;
-            default:
-                PyErr_Format(PyExc_ValueError,
-                  "unsupported format character '%c' (0x%x) "
-                  "at index " FORMAT_CODE_PY_SSIZE_T,
-                  c, c,
-                  (Py_ssize_t)(fmt - 1 - Bytes_AS_STRING(format)));
-                goto error;
-            }
-            if (width < len)
-                width = len;
-            if (rescnt < width) {
-                reslen -= rescnt;
-                rescnt = width + fmtcnt + 100;
-                reslen += rescnt;
-                if (reslen < 0) {
-                    Py_DECREF(result);
-                    Py_XDECREF(temp);
-                    if (args_owned)
-                        Py_DECREF(args);
-                    return PyErr_NoMemory();
-                }
-                if (!(result = resize_bytes(result, reslen))) {
-                    Py_XDECREF(temp);
-                    if (args_owned)
-                        Py_DECREF(args);
-                    return NULL;
-                }
-                res = Bytes_AS_STRING(result)
-                    + reslen - rescnt;
-            }
-            Py_MEMCPY(res, pbuf, len);
-            res += len;
-            rescnt -= len;
-            while (--width >= len) {
-                --rescnt;
-                *res++ = ' ';
-            }
-            if (dict && (argidx < arglen) && c != '%') {
-                PyErr_SetString(PyExc_TypeError,
-                           "not all arguments converted during string formatting");
-                Py_XDECREF(temp);
-                goto error;
-            }
-            Py_XDECREF(temp);
         } /* '%' */
     } /* until end */
-    if (argidx < arglen && !dict) {
-        PyErr_SetString(PyExc_TypeError,
-                        "not all arguments converted during string formatting");
+    
+    if (dict || (arglen < 0) || (argidx < 0)) {                                     // args' type is dict, the func ends
+        if (args_owned) Py_DECREF(args);
+        if (!(result = resize_bytes(result, reslen - rescnt))) return NULL;         // resize and return
+        if (place_holder != '%') {
+            PyErr_SetString(PyExc_TypeError, "place holder only expect %% when using dict");
+            goto error;
+        }
+        return result;
+    }
+
+    args_list = (char **)malloc(sizeof(char *) * arglen);                           // buffer
+    memset(args_list, 0, sizeof(char *) * arglen);
+    args_len = (Py_ssize_t *)malloc(sizeof(Py_ssize_t) * arglen);                   // length of every argument
+    while ((args_value = getnextarg(args, arglen, &argidx)) != NULL) {              // stop when receive NULL
+        Py_ssize_t length = 0;
+        if (!Bytes_CheckExact(args_value)) {
+            PyErr_Format(PyExc_ValueError, "only bytes values expected, got %s", Py_TYPE(args_value)->tp_name);  // may have bug
+            goto error;
+        }
+        Py_INCREF(args_value);                              // increase refcnt
+        args_buffer = Bytes_AS_STRING(args_value);
+        length = Bytes_GET_SIZE(args_value);
+        // printf("type: %s, len: %d, value: %s\n", Py_TYPE(args_value)->tp_name, length, args_buffer);
+        args_len[argidx - 1] = length;
+        args_list[argidx - 1] = (char *)malloc(sizeof(char) * (length + 1));
+        Py_MEMCPY(args_list[argidx - 1], args_buffer, length);
+        args_list[argidx - 1][length] = '\0';
+        Py_XDECREF(args_value);
+    }
+    
+    arg_usecnt = (int *)malloc(sizeof(int) * arglen);
+    memset(arg_usecnt, 0, sizeof(int) * arglen);
+
+    fmt = Bytes_AS_STRING(format);      // get pointer of format
+    fmtcnt = Bytes_GET_SIZE(format);    // get length of format
+    reslen = rescnt = 1;
+    while (reslen <= fmtcnt) {
+        reslen *= 2;
+        rescnt *= 2;
+    }
+    if ((result = resize_bytes(result, reslen)) == NULL) goto error;
+    res = Bytes_AS_STRING(result);
+    memset(res, 0, sizeof(char) * reslen);
+    
+    while (*fmt != '\0') {
+        if (*fmt != place_holder) {     // not place holder, pass it
+            if (!rescnt) {
+                rescnt += reslen;
+                reslen *= 2;
+                if ((result = resize_bytes(result, reslen)) == NULL) goto error;
+                res = Bytes_AS_STRING(result) + reslen - rescnt;
+            }
+            *(res++) = *(fmt++);
+            --rescnt;
+            continue;
+        }
+        if (*fmt == '%') {
+            char c = *(++fmt);
+            if (c == '\0') {            // if there is nothing after '%', raise an error
+                PyErr_SetString(PyExc_ValueError, "incomplete format");
+                goto error;
+            }
+            else if (c == '%') {        // '%%' will be transfered to '%'
+                if (!rescnt) {
+                    rescnt += reslen;
+                    reslen *= 2;
+                    if ((result = resize_bytes(result, reslen)) == NULL) goto error;
+                    res = Bytes_AS_STRING(result) + reslen - rescnt;
+                }
+                *res = c;
+                --rescnt;
+                ++res;
+                ++fmt;
+            }
+            else if (c == 's') {                // '%s', replace it with corresponding string
+                if (args_id >= arglen) {        // index is out of bound
+                    PyErr_SetString(PyExc_TypeError, "arguments not enough during string formatting");
+                    goto error;
+                }
+                if (rescnt < args_len[args_id]) {
+                    while (rescnt < args_len[args_id]) {
+                        rescnt += reslen;
+                        reslen *= 2;
+                    }
+                    if ((result = resize_bytes(result, reslen)) == NULL) goto error;
+                    res = Bytes_AS_STRING(result) + reslen - rescnt;
+                }
+                Py_MEMCPY(res, args_list[args_id], args_len[args_id]);
+                rescnt -= args_len[args_id];
+                res += args_len[args_id];
+                ++arg_usecnt[args_id];
+                ++args_id;
+                ++fmt;
+            }
+            else {                          // not support the character currently
+                PyErr_Format(PyExc_ValueError, "unsupported format character '%c' (0x%x) "
+                "at index " FORMAT_CODE_PY_SSIZE_T,
+                c, c,
+                (Py_ssize_t)(fmt - 1 - Bytes_AS_STRING(format)));
+                goto error;
+            }
+            continue;
+        }
+        if (*fmt == '$') {
+            char c = *(++fmt);
+            if (c == '\0') {            // if there is nothing after '$', raise an error
+                PyErr_SetString(PyExc_ValueError, "incomplete format");
+                goto error;
+            }
+            else if (c == '$') {        // '$$' will be transfered to'$'
+                if (!rescnt) {          // resize buffer
+                    rescnt += reslen;
+                    reslen *= 2;
+                    if ((result = resize_bytes(result, reslen)) == NULL) goto error;
+                    res = Bytes_AS_STRING(result) + reslen - rescnt;
+                }
+                *res = c;
+                --rescnt;
+                ++res;
+                ++fmt;
+            }
+            else if (isdigit(c)) {      // represents '$number'
+                int index = 0;
+                index_type = 1;
+                while (isdigit(*fmt)) {
+                    index = index * 10 + (*fmt) -'0';
+                    ++fmt;
+                }
+                if ((index > arglen) || (index <= 0)) {       // invalid index
+                    PyErr_SetString(PyExc_ValueError, "invalid index");
+                    goto error;
+                }
+                --index;
+                if (rescnt < args_len[index]) {
+                    while (rescnt < args_len[index]) {
+                        rescnt += reslen;
+                        reslen *= 2;
+                    }
+                    if ((result = resize_bytes(result, reslen)) == NULL) goto error;
+                    res = Bytes_AS_STRING(result) + reslen - rescnt;
+                }
+                Py_MEMCPY(res, args_list[index], args_len[index]);
+                rescnt -= args_len[index];
+                res += args_len[index];
+                ++arg_usecnt[index];
+            }
+            else {                      // invalid place holder
+                PyErr_Format(PyExc_ValueError, "unsupported format character '%c' (0x%x) "
+                "at index " FORMAT_CODE_PY_SSIZE_T,
+                c, c,
+                (Py_ssize_t)(fmt - 1 - Bytes_AS_STRING(format)));
+                goto error;
+            }
+        }
+    }
+    if ((args_id < arglen) && (!dict) && (!index_type)) {       // not all arguments are used, '%' type
+        PyErr_SetString(PyExc_TypeError, "not all arguments converted during string formatting");
         goto error;
     }
-    if (args_owned) {
-        Py_DECREF(args);
+    for (args_id = 0; args_id < arglen; ++args_id) {            // not all arguments are used, '$' type
+        if (!arg_usecnt[args_id]) {
+            PyErr_SetString(PyExc_TypeError, "not all arguments converted during string formatting");
+            goto error;
+        }
     }
-    if (!(result = resize_bytes(result, reslen - rescnt))) {
-        return NULL;
+    if (args_list != NULL) {
+        while (--argidx >= 0) free(args_list[argidx]);
+        free(args_list), free(args_len), free(arg_usecnt);
     }
+    if (args_owned) Py_DECREF(args);
+    if (!(result = resize_bytes(result, reslen - rescnt))) return NULL;        // resize
     return result;
 
- error:
-    Py_DECREF(result);
-    if (args_owned) {
-        Py_DECREF(args);
+error:
+    if (args_list != NULL) {    // release all the refcnt
+        while (--argidx >= 0) free(args_list[argidx]);
+        free(args_list), free(args_len), free(arg_usecnt);
     }
+    Py_DECREF(result);
+    if (args_owned) Py_DECREF(args);
     return NULL;
 }

psycopg/connection.h

@@ -46,6 +46,13 @@ extern "C" {
 #define STATE_ON                1
 #define STATE_DEFAULT           2
 
+/* sql_compatibility values */
+#define SQL_COMPATIBILITY_A         1
+#define SQL_COMPATIBILITY_OTHER     5
+// #define SQL_COMPATIBILITY_B     2
+// #define SQL_COMPATIBILITY_C     3
+// #define SQL_COMPATIBILITY_PG    4
+
 /* connection status */
 #define CONN_STATUS_SETUP       0
 #define CONN_STATUS_READY       1
@@ -148,6 +155,8 @@ struct connectionObject {
 
     /* inside a with block */
     int entered;
+
+    int sql_compatibility;
 };
 
 /* map isolation level values into a numeric const */

psycopg/connection_type.c

@@ -37,6 +37,8 @@
 
 #include <string.h>
 #include <ctype.h>
+#include "psycopg/typecast.h"
+#include "psycopg/typecast_basic.c"
 
 extern HIDDEN const char *srv_isolevels[];
 extern HIDDEN const char *srv_readonly[];
@@ -1268,6 +1270,9 @@ static struct PyMemberDef connectionObject_members[] = {
     {"server_version", T_INT,
         offsetof(connectionObject, server_version), READONLY,
         "Server version."},
+    {"sql_compatibility", T_INT,
+        offsetof(connectionObject, sql_compatibility), READONLY,
+        "Server sql_compatibility param value."},
     {NULL}
 };
 
@@ -1311,12 +1316,61 @@ static struct PyGetSetDef connectionObject_getsets[] = {
 };
 #undef EXCEPTION_GETTER
 
+
+/* register the uint typecasters */
+static int
+register_type_uint(connectionObject *self, PyThreadState **tstate)
+{
+    int rv = -1;
+    typecastObject *obj = NULL;
+    PyObject *name = NULL, *values = NULL;
+    Py_ssize_t i, len = 0;
+
+    char* uint_arr[] = {"\'uint1\'", "\'uint2\'", "\'uint4\'", "\'uint8\'"};
+    int size = sizeof(uint_arr) / sizeof(uint_arr[0]);
+    long int* _typecast_INTEGER_types;
+    _typecast_INTEGER_types = (long int*)malloc((size+1)*sizeof(long int));
+
+    for (int i=0; i< size; i++) {
+        unsigned int uint_val = pq_get_pg_catalog_custom_type_oid(self, uint_arr[i], tstate);
+        _typecast_INTEGER_types[i] = (long int)uint_val;
+    }
+    typecastObject_initlist _typecast_builtins[] = {
+        {"INTEGER", _typecast_INTEGER_types, typecast_INTEGER_cast, NULL},
+    };
+    name = Text_FromUTF8(_typecast_builtins->name);
+    if (!name) goto end;
+
+    while (_typecast_builtins->values[len] != 0) len++;
+
+    values = PyTuple_New(len);
+    if (!values) goto end;
+
+    for (i = 0; i < len ; i++) {
+        PyTuple_SET_ITEM(values, i, PyInt_FromLong(_typecast_builtins->values[i]));
+    }
+    obj = (typecastObject *)typecast_new(name, values, NULL, NULL);
+    if (obj) {
+        obj->ccast = _typecast_builtins->cast;
+        obj->pcast = NULL;
+    }
+    if (typecast_add((PyObject *)obj, self->string_types, 0) < 0) { goto end; }
+    rv = 0;
+    free(_typecast_INTEGER_types);
+end:
+    Py_XDECREF(values);
+    Py_XDECREF(name);
+    return rv;
+}
+
+
 /* initialization and finalization methods */
 
 static int
 connection_setup(connectionObject *self, const char *dsn, long int async)
 {
     int rv = -1;
+    char *sql_compatibility_value = NULL;
 
     Dprintf("connection_setup: init connection object at %p, "
 	    "async %ld, refcnt = " FORMAT_CODE_PY_SSIZE_T,
@@ -1334,6 +1388,7 @@ connection_setup(connectionObject *self, const char *dsn, long int async)
     self->isolevel = ISOLATION_LEVEL_DEFAULT;
     self->readonly = STATE_DEFAULT;
     self->deferrable = STATE_DEFAULT;
+    self->sql_compatibility = SQL_COMPATIBILITY_A;
 #ifdef CONN_CHECK_PID
     self->procpid = getpid();
 #endif
@@ -1356,7 +1411,23 @@ connection_setup(connectionObject *self, const char *dsn, long int async)
         FORMAT_CODE_PY_SSIZE_T,
         self, Py_REFCNT(self));
 
+    Py_BEGIN_ALLOW_THREADS;
+    pthread_mutex_lock(&self->lock);
+    sql_compatibility_value = pq_get_guc_locked(self, "sql_compatibility", &_save);
+    if (register_type_uint(self, &_save)) { goto exit; }
+    pthread_mutex_unlock(&self->lock);
+    Py_END_ALLOW_THREADS;
+
+    if (strcmp(sql_compatibility_value, "A") == 0) {
+        self->sql_compatibility = SQL_COMPATIBILITY_A;
+    } else {
+        self->sql_compatibility = SQL_COMPATIBILITY_OTHER;
+    }
+
 exit:
+    if (sql_compatibility_value){
+        free(sql_compatibility_value);
+    }
     return rv;
 }
 

psycopg/cursor_type.c

@@ -127,12 +127,13 @@ exit:
 /* mogrify a query string and build argument array or dict */
 
 RAISES_NEG static int
-_mogrify(PyObject *var, PyObject *fmt, cursorObject *curs, PyObject **new)
+_mogrify(PyObject *var, PyObject *fmt, cursorObject *curs, PyObject **new, char place_holder)
 {
     PyObject *key, *value, *n;
     const char *d, *c;
     Py_ssize_t index = 0;
     int force = 0, kind = 0;
+    int max_index = 0;
 
     /* from now on we'll use n and replace its value in *new only at the end,
        just before returning. we also init *new to NULL to exit with an error
@@ -141,164 +142,199 @@ _mogrify(PyObject *var, PyObject *fmt, cursorObject *curs, PyObject **new)
     c = Bytes_AsString(fmt);
 
     while(*c) {
-        if (*c++ != '%') {
-            /* a regular character */
-            continue;
-        }
-
-        switch (*c) {
-
-        /* handle plain percent symbol in format string */
-        case '%':
+        while ((*c != '\0') && (*c != place_holder)) ++c;
+        if (*c == '%') {
             ++c;
             force = 1;
-            break;
+            if (*c == '(') {
+                if ((kind == 2) || (kind == 3)) {
+                    Py_XDECREF(n);
+                    psyco_set_error(ProgrammingError, curs,
+                    "argument formats can't be mixed");
+                    return -1;
+                }
+                kind = 1;
 
-        /* if we find '%(' then this is a dictionary, we:
-           1/ find the matching ')' and extract the key name
-           2/ locate the value in the dictionary (or return an error)
-           3/ mogrify the value into something useful (quoting)...
-           4/ ...and add it to the new dictionary to be used as argument
-        */
-        case '(':
-            /* check if some crazy guy mixed formats */
-            if (kind == 2) {
-                Py_XDECREF(n);
-                psyco_set_error(ProgrammingError, curs,
-                   "argument formats can't be mixed");
-                return -1;
+                /* let's have d point the end of the argument */
+                for (d = c + 1; *d && *d != ')' && *d != '%'; d++);
+
+                if (*d == ')') {
+                    if (!(key = Text_FromUTF8AndSize(c+1, (Py_ssize_t)(d-c-1)))) {
+                        Py_XDECREF(n);
+                        return -1;
+                    }
+
+                    /*  if value is NULL we did not find the key (or this is not a
+                        dictionary): let python raise a KeyError */
+                    if (!(value = PyObject_GetItem(var, key))) {
+                        Py_DECREF(key); /* destroy key */
+                        Py_XDECREF(n);  /* destroy n */
+                        return -1;
+                    }
+                    /* key has refcnt 1, value the original value + 1 */
+
+                    Dprintf("_mogrify: value refcnt: "
+                    FORMAT_CODE_PY_SSIZE_T " (+1)", Py_REFCNT(value));
+
+                    if (n == NULL) {
+                        if (!(n = PyDict_New())) {
+                            Py_DECREF(key);
+                            Py_DECREF(value);
+                            return -1;
+                        }
+                    }
+
+                    if (0 == PyDict_Contains(n, key)) {
+                        PyObject *t = NULL;
+
+                        /* None is always converted to NULL; this is an
+                        optimization over the adapting code and can go away in
+                        the future if somebody finds a None adapter useful. */
+                        if (value == Py_None) {
+                            Py_INCREF(psyco_null);
+                            t = psyco_null;
+                            PyDict_SetItem(n, key, t);
+                            /* t is a new object, refcnt = 1, key is at 2 */
+                        }
+                        else {
+                            t = microprotocol_getquoted(value, curs->conn);
+                            if (t != NULL) {
+                                PyDict_SetItem(n, key, t);
+                                /* both key and t refcnt +1, key is at 2 now */
+                            }
+                            else {
+                                /* no adapter found, raise a BIG exception */
+                                Py_DECREF(key);
+                                Py_DECREF(value);
+                                Py_DECREF(n);
+                                return -1;
+                            }
+                        }
+
+                        Py_XDECREF(t); /* t dies here */
+                    }
+                    Py_DECREF(value);
+                    Py_DECREF(key); /* key has the original refcnt now */
+                    Dprintf("_mogrify: after value refcnt: "
+                        FORMAT_CODE_PY_SSIZE_T, Py_REFCNT(value));
+                }
+                else {
+                    /* we found %( but not a ) */
+                    Py_XDECREF(n);
+                    psyco_set_error(ProgrammingError, curs,
+                    "incomplete placeholder: '%(' without ')'");
+                    return -1;
+                }
+                c = d + 1;
             }
-            kind = 1;
+            else if (*c == 's') {
+                /* this is a format that expects a tuple; it is much easier,
+                because we don't need to check the old/new dictionary for
+                keys */
 
-            /* let's have d point the end of the argument */
-            for (d = c + 1; *d && *d != ')' && *d != '%'; d++);
+                /* check if some crazy guy mixed formats */
+                if ((kind == 1) || (kind == 3)) {
+                    Py_XDECREF(n);
+                    psyco_set_error(ProgrammingError, curs,
+                    "argument formats can't be mixed");
+                    return -1;
+                }
+                kind = 2;
 
-            if (*d == ')') {
-                if (!(key = Text_FromUTF8AndSize(c+1, (Py_ssize_t)(d-c-1)))) {
+                value = PySequence_GetItem(var, index);
+                /* value has refcnt inc'ed by 1 here */
+
+                /*  if value is NULL this is not a sequence or the index is wrong;
+                    anyway we let python set its own exception */
+                if (value == NULL) {
                     Py_XDECREF(n);
                     return -1;
                 }
 
-                /*  if value is NULL we did not find the key (or this is not a
-                    dictionary): let python raise a KeyError */
-                if (!(value = PyObject_GetItem(var, key))) {
-                    Py_DECREF(key); /* destroy key */
-                    Py_XDECREF(n);  /* destroy n */
-                    return -1;
-                }
-                /* key has refcnt 1, value the original value + 1 */
-
-                Dprintf("_mogrify: value refcnt: "
-                  FORMAT_CODE_PY_SSIZE_T " (+1)", Py_REFCNT(value));
-
                 if (n == NULL) {
-                    if (!(n = PyDict_New())) {
-                        Py_DECREF(key);
+                    if (!(n = PyTuple_New(PyObject_Length(var)))) {
                         Py_DECREF(value);
                         return -1;
                     }
                 }
 
-                if (0 == PyDict_Contains(n, key)) {
-                    PyObject *t = NULL;
-
-                    /* None is always converted to NULL; this is an
-                       optimization over the adapting code and can go away in
-                       the future if somebody finds a None adapter useful. */
-                    if (value == Py_None) {
-                        Py_INCREF(psyco_null);
-                        t = psyco_null;
-                        PyDict_SetItem(n, key, t);
-                        /* t is a new object, refcnt = 1, key is at 2 */
-                    }
-                    else {
-                        t = microprotocol_getquoted(value, curs->conn);
-                        if (t != NULL) {
-                            PyDict_SetItem(n, key, t);
-                            /* both key and t refcnt +1, key is at 2 now */
-                        }
-                        else {
-                            /* no adapter found, raise a BIG exception */
-                            Py_DECREF(key);
-                            Py_DECREF(value);
-                            Py_DECREF(n);
-                            return -1;
-                        }
-                    }
-
-                    Py_XDECREF(t); /* t dies here */
-                }
-                Py_DECREF(value);
-                Py_DECREF(key); /* key has the original refcnt now */
-                Dprintf("_mogrify: after value refcnt: "
-                    FORMAT_CODE_PY_SSIZE_T, Py_REFCNT(value));
-            }
-            else {
-                /* we found %( but not a ) */
-                Py_XDECREF(n);
-                psyco_set_error(ProgrammingError, curs,
-                   "incomplete placeholder: '%(' without ')'");
-                return -1;
-            }
-            c = d + 1;  /* after the ) */
-            break;
-
-        default:
-            /* this is a format that expects a tuple; it is much easier,
-               because we don't need to check the old/new dictionary for
-               keys */
-
-            /* check if some crazy guy mixed formats */
-            if (kind == 1) {
-                Py_XDECREF(n);
-                psyco_set_error(ProgrammingError, curs,
-                  "argument formats can't be mixed");
-                return -1;
-            }
-            kind = 2;
-
-            value = PySequence_GetItem(var, index);
-            /* value has refcnt inc'ed by 1 here */
-
-            /*  if value is NULL this is not a sequence or the index is wrong;
-                anyway we let python set its own exception */
-            if (value == NULL) {
-                Py_XDECREF(n);
-                return -1;
-            }
-
-            if (n == NULL) {
-                if (!(n = PyTuple_New(PyObject_Length(var)))) {
-                    Py_DECREF(value);
-                    return -1;
-                }
-            }
-
-            /* let's have d point just after the '%' */
-            if (value == Py_None) {
-                Py_INCREF(psyco_null);
-                PyTuple_SET_ITEM(n, index, psyco_null);
-                Py_DECREF(value);
-            }
-            else {
-                PyObject *t = microprotocol_getquoted(value, curs->conn);
-
-                if (t != NULL) {
-                    PyTuple_SET_ITEM(n, index, t);
+                /* let's have d point just after the '%' */
+                if (value == Py_None) {
+                    Py_INCREF(psyco_null);
+                    PyTuple_SET_ITEM(n, index, psyco_null);
                     Py_DECREF(value);
                 }
                 else {
-                    Py_DECREF(n);
-                    Py_DECREF(value);
+                    PyObject *t = microprotocol_getquoted(value, curs->conn);
+
+                    if (t != NULL) {
+                        PyTuple_SET_ITEM(n, index, t);
+                        Py_DECREF(value);
+                    }
+                    else {
+                        Py_DECREF(n);
+                        Py_DECREF(value);
+                        return -1;
+                    }
+                }
+                index += 1;
+            }
+        }
+
+        else if (*c == '$') {       // new place holder $
+            int tmp_index = 0;
+            if ((kind == 1) || (kind == 2)) {
+                Py_XDECREF(n);
+                psyco_set_error(ProgrammingError, curs,
+                "argument formats can't be mixed");
+                return -1;
+            }
+            kind = 3;               // kind = 3 means using 
+
+            ++c;
+            while (isdigit(*c)) {   // calculate index
+                tmp_index = tmp_index * 10 + (*c) -'0';
+                ++c;
+            }
+            --tmp_index;
+
+            for (; max_index <= tmp_index; ++max_index) {
+                // to avoid index not cover all arguments, which may cause double free in bytes_format
+                int id = max_index;
+                value = PySequence_GetItem(var, id);
+                if (value == NULL) {
+                    Py_XDECREF(n);
                     return -1;
                 }
+                if (n == NULL) {
+                    if (!(n = PyTuple_New(PyObject_Length(var)))) {
+                        Py_DECREF(value);
+                        return -1;
+                    }
+                }
+                if (value == Py_None) {
+                    Py_INCREF(psyco_null);
+                    PyTuple_SET_ITEM(n, id, psyco_null);
+                    Py_DECREF(value);
+                }
+                else {
+                    PyObject *t = microprotocol_getquoted(value, curs->conn);
+                    if (t != NULL) {
+                        PyTuple_SET_ITEM(n, id, t);
+                        Py_DECREF(value);
+                    }
+                    else {
+                        Py_DECREF(n);
+                        Py_DECREF(value);
+                        return -1;
+                    }
+                }
             }
-            index += 1;
+            
         }
     }
 
-    if (force && n == NULL)
-        n = PyTuple_New(0);
+    if (force && n == NULL) n = PyTuple_New(0);
     *new = n;
 
     return 0;
@@ -314,7 +350,7 @@ _mogrify(PyObject *var, PyObject *fmt, cursorObject *curs, PyObject **new)
  */
 static PyObject *
 _psyco_curs_merge_query_args(cursorObject *self,
-                             PyObject *query, PyObject *args)
+                             PyObject *query, PyObject *args, char place_holder)
 {
     PyObject *fquery;
 
@@ -329,7 +365,7 @@ _psyco_curs_merge_query_args(cursorObject *self,
        the current exception (we will later restore it if the type or the
        strings do not match.) */
 
-    if (!(fquery = Bytes_Format(query, args))) {
+    if (!(fquery = Bytes_Format(query, args, place_holder))) {
         PyObject *err, *arg, *trace;
         int pe = 0;
 
@@ -376,7 +412,7 @@ _psyco_curs_merge_query_args(cursorObject *self,
 RAISES_NEG static int
 _psyco_curs_execute(cursorObject *self,
                     PyObject *query, PyObject *vars,
-                    long int async, int no_result)
+                    char place_holder, long int async, int no_result)
 {
     int res = -1;
     int tmp;
@@ -396,12 +432,12 @@ _psyco_curs_execute(cursorObject *self,
        the right thing (i.e., what the user expects) */
     if (vars && vars != Py_None)
     {
-        if (0 > _mogrify(vars, query, self, &cvt)) { goto exit; }
+        if (0 > _mogrify(vars, query, self, &cvt, place_holder)) { goto exit; }
     }
 
     /* Merge the query to the arguments if needed */
     if (cvt) {
-        if (!(fquery = _psyco_curs_merge_query_args(self, query, cvt))) {
+        if (!(fquery = _psyco_curs_merge_query_args(self, query, cvt, place_holder))) {
             goto exit;
         }
     }
@@ -461,15 +497,28 @@ exit:
 static PyObject *
 curs_execute(cursorObject *self, PyObject *args, PyObject *kwargs)
 {
-    PyObject *vars = NULL, *operation = NULL;
+    PyObject *vars = NULL, *operation = NULL, *Place_holder = NULL;
+    char place_holder = '%';    //default value: '%'
 
-    static char *kwlist[] = {"query", "vars", NULL};
+    static char *kwlist[] = {"query", "vars", "place_holder", NULL};
 
-    if (!PyArg_ParseTupleAndKeywords(args, kwargs, "O|O", kwlist,
-                                     &operation, &vars)) {
+    if (!PyArg_ParseTupleAndKeywords(args, kwargs, "O|OO", kwlist,
+                                     &operation, &vars, &Place_holder)) {
         return NULL;
     }
 
+    if (Place_holder != NULL) { //if exists place holder argument, it will be checked and parse
+        if (!(Place_holder = curs_validate_sql_basic(self, Place_holder))) {
+            psyco_set_error(ProgrammingError, self, "can't parse place holder");
+            return NULL;
+        }
+        if (Bytes_GET_SIZE(Place_holder) != 1) {
+            psyco_set_error(ProgrammingError, self, "place holder must be a character");
+            return NULL;
+        }
+        place_holder = Bytes_AS_STRING(Place_holder)[0];
+    }
+
     if (self->name != NULL) {
         if (self->query) {
             psyco_set_error(ProgrammingError, self,
@@ -488,7 +537,7 @@ curs_execute(cursorObject *self, PyObject *args, PyObject *kwargs)
     EXC_IF_ASYNC_IN_PROGRESS(self, execute);
     EXC_IF_TPC_PREPARED(self->conn, execute);
 
-    if (0 > _psyco_curs_execute(self, operation, vars, self->conn->async, 0)) {
+    if (0 > _psyco_curs_execute(self, operation, vars, place_holder, self->conn->async, 0)) {
         return NULL;
     }
 
@@ -502,20 +551,33 @@ curs_execute(cursorObject *self, PyObject *args, PyObject *kwargs)
 static PyObject *
 curs_executemany(cursorObject *self, PyObject *args, PyObject *kwargs)
 {
-    PyObject *operation = NULL, *vars = NULL;
+    PyObject *operation = NULL, *vars = NULL, *Place_holder = NULL;
     PyObject *v, *iter = NULL;
+    char place_holder = '%';
     long rowcount = 0;
 
-    static char *kwlist[] = {"query", "vars_list", NULL};
+    static char *kwlist[] = {"query", "vars_list", "plae_holder", NULL};
 
     /* reset rowcount to -1 to avoid setting it when an exception is raised */
     self->rowcount = -1;
 
-    if (!PyArg_ParseTupleAndKeywords(args, kwargs, "OO", kwlist,
-                                     &operation, &vars)) {
+    if (!PyArg_ParseTupleAndKeywords(args, kwargs, "O|OO", kwlist,
+                                     &operation, &vars, &Place_holder)) {
         return NULL;
     }
 
+    if (Place_holder != NULL) {
+        if (!(Place_holder = curs_validate_sql_basic(self, Place_holder))) {
+            psyco_set_error(ProgrammingError, self, "can't parse place holder");
+            return NULL;
+        }
+        if (Bytes_GET_SIZE(Place_holder) != 1) {
+            psyco_set_error(ProgrammingError, self, "place holder must be a character");
+            return NULL;
+        }
+        place_holder = Bytes_AS_STRING(Place_holder)[0];
+    }
+
     EXC_IF_CURS_CLOSED(self);
     EXC_IF_CURS_ASYNC(self, executemany);
     EXC_IF_TPC_PREPARED(self->conn, executemany);
@@ -532,7 +594,7 @@ curs_executemany(cursorObject *self, PyObject *args, PyObject *kwargs)
     }
 
     while ((v = PyIter_Next(vars)) != NULL) {
-        if (0 > _psyco_curs_execute(self, operation, v, 0, 1)) {
+        if (0 > _psyco_curs_execute(self, operation, v, place_holder, 0, 1)) {
             Py_DECREF(v);
             Py_XDECREF(iter);
             return NULL;
@@ -556,13 +618,225 @@ curs_executemany(cursorObject *self, PyObject *args, PyObject *kwargs)
     }
 }
 
+#define curs_execute_prepared_batch_doc \
+"execute_prepared_batch(statement_name, nParams, nBatch, varsList) -- Execute prepared sql with bound batch vars"
+
+static PyObject *
+curs_execute_prepared_batch(cursorObject *self, PyObject *args)
+{
+    const char *stmtName = NULL;
+    int nParams = 0, nBatch = 0;
+    PyObject *argsList = NULL;
+
+    int rowIdx, colIdx, total;
+    char **paramValues = NULL;
+    PGresult *res = NULL;
+
+    /* reset rowcount to -1 to avoid setting it when an exception is raised */
+    self->rowcount = -1;
+
+    if (!PyArg_ParseTuple(args, "siiO", &stmtName, &nParams, &nBatch, &argsList))
+    {
+        Dprintf("execute_prepared_batch: parse tuple failed");
+        goto exit;
+    }
+    Dprintf("execute_prepared_batch parsed statement_name: %s, nParams: %d, nBatch: %d",
+            stmtName, nParams, nBatch);
+    total = nBatch * nParams;
+
+    EXC_IF_CURS_CLOSED(self);
+    EXC_IF_CURS_ASYNC(self, execute_prepared_batch);
+    EXC_IF_TPC_PREPARED(self->conn, execute_prepared_batch);
+
+    if (self->name != NULL) {
+        psyco_set_error(ProgrammingError, self, "can't call .execute_prepared_batch() on named cursors");
+        goto exit;
+    }
+
+    // allocate all memory
+    if (!(paramValues = (char**)malloc(sizeof(char*) * total))) {
+        PyErr_NoMemory();
+        goto exit;
+    }
+    memset(paramValues, 0, sizeof(char *) * total);
+
+    if (!PySequence_Check(argsList)) {
+        psyco_set_error(DataError, self, "expect varsList is a list");
+        goto exit;
+    }
+
+    for (rowIdx = 0; rowIdx < nBatch; rowIdx++) {
+        PyObject *rowArgs = PySequence_GetItem(argsList, rowIdx);
+
+        // Check if the inner object is a list
+        if (!PySequence_Check(rowArgs)) {
+            psyco_set_error(DataError, self, "expect every item in varsList is a list");
+            goto exit;
+        }
+
+        // Loop through each row of parameters
+        for (colIdx = 0; colIdx < nParams; colIdx++) {
+            PyObject *argItem = PySequence_GetItem(rowArgs, colIdx);
+
+            if (argItem == Py_None) {
+                paramValues[rowIdx * nParams + colIdx] = NULL;
+            } else {
+                if (!(argItem = psyco_ensure_bytes(argItem))) {
+                    goto exit;
+                }
+                // convert empty string to NULL in A compatibility mode
+                if (self->conn->sql_compatibility == SQL_COMPATIBILITY_A && PyObject_Length(argItem) == 0) {
+                    paramValues[rowIdx * nParams + colIdx] = NULL;
+                } else {
+                    paramValues[rowIdx * nParams + colIdx] = Bytes_AsString(argItem);
+                }
+            }
+            Py_XDECREF(argItem);
+        }
+        Py_XDECREF(rowArgs);
+    }
+
+    res = PQexecPreparedBatch(self->conn->pgconn, stmtName, nParams, nBatch,
+                                paramValues, NULL, NULL, 0);
+    conn_set_result(self->conn, res);
+    if (PQresultStatus(res) != PGRES_COMMAND_OK) {
+        Dprintf("execute_prepared_batch error: %s", PQresultErrorMessage(res));
+        psyco_set_error(OperationalError, self, PQresultErrorMessage(res));
+        goto exit;
+    }
+
+exit:
+    free(paramValues);
+    Py_RETURN_NONE;
+}
+
+
+#define curs_execute_params_batch_doc \
+"execute_params_batch(sql, nParams, nBatch, varsList) -- Execute sql with bound batch vars"
+
+static PyObject *
+curs_execute_params_batch(cursorObject *self, PyObject *args)
+{
+    const char *sql = NULL;
+    int nParams = 0, nBatch = 0;
+    PyObject *argsList = NULL;
+
+    int rowIdx, colIdx, total;
+    char **paramValues = NULL;
+    PGresult *res = NULL;
+
+    self->rowcount = -1;
+
+    if (!PyArg_ParseTuple(args, "siiO", &sql, &nParams, &nBatch, &argsList))
+    {
+        Dprintf("execute_params_batch: parse tuple failed");
+        goto exit;
+    }
+    Dprintf("execute_params_batch parsed sql: %s, nParams: %d, nBatch: %d",
+            sql, nParams, nBatch);
+
+    total = nBatch * nParams;
+
+    EXC_IF_CURS_CLOSED(self);
+    EXC_IF_CURS_ASYNC(self, execute_params_batch);
+    EXC_IF_TPC_PREPARED(self->conn, execute_params_batch);
+
+    if (self->name != NULL) {
+        psyco_set_error(ProgrammingError, self, "can't call .execute_params_batch() on named cursors");
+        goto exit;
+    }
+
+    if (!(paramValues = (char**)malloc(sizeof(char*) * total))) {
+        PyErr_NoMemory();
+        goto exit;
+    }
+    memset(paramValues, 0, sizeof(char *) * total);
+
+    if (!PySequence_Check(argsList)) {
+        psyco_set_error(DataError, self, "expect varsList is a list");
+        goto exit;
+    }
+
+    for (rowIdx = 0; rowIdx < nBatch; rowIdx++) {
+        PyObject *rowArgs = PySequence_GetItem(argsList, rowIdx);
+
+        // Check if the inner object is a list
+        if (!PySequence_Check(rowArgs)) {
+            psyco_set_error(DataError, self, "expect every item in varsList is a list");
+            goto exit;
+        }
+
+        // Loop through each row of parameters
+        for (colIdx = 0; colIdx < nParams; colIdx++) {
+            PyObject *argItem = PySequence_GetItem(rowArgs, colIdx);
+
+            if (argItem == Py_None) {
+                paramValues[rowIdx * nParams + colIdx] = NULL;
+            } else {
+                if (!(argItem = psyco_ensure_bytes(argItem))) {
+                    goto exit;
+                }
+                // convert empty string to NULL in A compatibility mode
+                if (self->conn->sql_compatibility == SQL_COMPATIBILITY_A && PyObject_Length(argItem) == 0) {
+                    paramValues[rowIdx * nParams + colIdx] = NULL;
+                } else {
+                    paramValues[rowIdx * nParams + colIdx] = Bytes_AsString(argItem);
+                }
+            }
+            Py_XDECREF(argItem);
+        }
+        Py_XDECREF(rowArgs);
+    }
+
+    res = PQexecParamsBatch(self->conn->pgconn, sql, nParams, nBatch, NULL,
+                            paramValues, NULL, NULL, 0);
+    conn_set_result(self->conn, res);
+    if (PQresultStatus(res) != PGRES_COMMAND_OK) {
+        Dprintf("execute_params_batch error: %s", PQresultErrorMessage(res));
+        psyco_set_error(OperationalError, self, PQresultErrorMessage(res));
+        goto exit;
+    }
+
+exit:
+    free(paramValues);
+    Py_RETURN_NONE;
+}
+
+#define curs_prepare_doc \
+"prepare(name, command, nparams) -- Prepare a statement"
+
+static PyObject *
+curs_prepare(cursorObject *self, PyObject *args)
+{
+    const char *stmtname = NULL;
+    const char *query = NULL;
+    int nparams;
+    PGresult *res = NULL;
+
+    if (!PyArg_ParseTuple(args, "ssi", &stmtname, &query, &nparams)) {
+        return NULL;
+    }
+
+    EXC_IF_CURS_CLOSED(self);
+    EXC_IF_ASYNC_IN_PROGRESS(self, prepare);
+    EXC_IF_TPC_PREPARED(self->conn, prepare);
+
+    res = PQprepare(self->conn->pgconn, stmtname, query, nparams, NULL);
+    conn_set_result(self->conn, res);
+    if (PQresultStatus(res) != PGRES_COMMAND_OK) {
+        psyco_set_error(OperationalError, self, PQresultErrorMessage(res));
+        return NULL;
+    }
+
+    Py_RETURN_NONE;
+}
 
 #define curs_mogrify_doc \
 "mogrify(query, vars=None) -> str -- Return query after vars binding."
 
 static PyObject *
 _psyco_curs_mogrify(cursorObject *self,
-                   PyObject *operation, PyObject *vars)
+                   PyObject *operation, PyObject *vars, char place_holder)
 {
     PyObject *fquery = NULL, *cvt = NULL;
 
@@ -577,13 +851,13 @@ _psyco_curs_mogrify(cursorObject *self,
 
     if (vars && vars != Py_None)
     {
-        if (0 > _mogrify(vars, operation, self, &cvt)) {
+        if (0 > _mogrify(vars, operation, self, &cvt, place_holder)) {
             goto cleanup;
         }
     }
 
     if (vars && cvt) {
-        if (!(fquery = _psyco_curs_merge_query_args(self, operation, cvt))) {
+        if (!(fquery = _psyco_curs_merge_query_args(self, operation, cvt, place_holder))) {
             goto cleanup;
         }
 
@@ -606,16 +880,29 @@ cleanup:
 static PyObject *
 curs_mogrify(cursorObject *self, PyObject *args, PyObject *kwargs)
 {
-    PyObject *vars = NULL, *operation = NULL;
+    PyObject *vars = NULL, *operation = NULL, *Place_holder = NULL;
+    char place_holder = '%';
 
-    static char *kwlist[] = {"query", "vars", NULL};
+    static char *kwlist[] = {"query", "vars", "place_holder", NULL};
 
-    if (!PyArg_ParseTupleAndKeywords(args, kwargs, "O|O", kwlist,
-                                     &operation, &vars)) {
+    if (!PyArg_ParseTupleAndKeywords(args, kwargs, "O|OO", kwlist,
+                                     &operation, &vars, &Place_holder)) {
         return NULL;
     }
 
-    return _psyco_curs_mogrify(self, operation, vars);
+    if (Place_holder != NULL) {
+        if (!(Place_holder = curs_validate_sql_basic(self, Place_holder))) {
+            psyco_set_error(ProgrammingError, self, "can't parse place holder");
+            return NULL;
+        }
+        if (Bytes_GET_SIZE(Place_holder) != 1) {
+            psyco_set_error(ProgrammingError, self, "place holder must be a character");
+            return NULL;
+        }
+        place_holder = Bytes_AS_STRING(Place_holder)[0];
+    }
+
+    return _psyco_curs_mogrify(self, operation, vars, place_holder);
 }
 
 
@@ -1016,11 +1303,25 @@ curs_callproc(cursorObject *self, PyObject *args)
     PyObject *pvals = NULL;
     char *cpname = NULL;
     char **scpnames = NULL;
+    PyObject *Place_holder = NULL;
+    char place_holder = '%';
 
-    if (!PyArg_ParseTuple(args, "s#|O", &procname, &procname_len,
-                &parameters)) {
+    if (!PyArg_ParseTuple(args, "s#|OO", &procname, &procname_len,
+                &parameters, &Place_holder)) {
         goto exit;
     }
+    
+    if (Place_holder != NULL) {
+        if (!(Place_holder = curs_validate_sql_basic(self, Place_holder))) {
+            psyco_set_error(ProgrammingError, self, "can't parse place holder");
+            return NULL;
+        }
+        if (Bytes_GET_SIZE(Place_holder) != 1) {
+            psyco_set_error(ProgrammingError, self, "place holder must be a character");
+            return NULL;
+        }
+        place_holder = Bytes_AS_STRING(Place_holder)[0];
+    }
 
     EXC_IF_CURS_CLOSED(self);
     EXC_IF_ASYNC_IN_PROGRESS(self, callproc);
@@ -1114,7 +1415,7 @@ curs_callproc(cursorObject *self, PyObject *args)
     }
 
     if (0 <= _psyco_curs_execute(
-            self, operation, pvals, self->conn->async, 0)) {
+            self, operation, pvals, place_holder, self->conn->async, 0)) {
         /* The dict case is outside DBAPI scope anyway, so simply return None */
         if (using_dict) {
             res = Py_None;
@@ -1824,6 +2125,12 @@ static struct PyMethodDef cursorObject_methods[] = {
      METH_VARARGS|METH_KEYWORDS, curs_execute_doc},
     {"executemany", (PyCFunction)curs_executemany,
      METH_VARARGS|METH_KEYWORDS, curs_executemany_doc},
+    {"execute_prepared_batch", (PyCFunction)curs_execute_prepared_batch,
+    METH_VARARGS, curs_execute_prepared_batch_doc},
+    {"execute_params_batch", (PyCFunction)curs_execute_params_batch,
+    METH_VARARGS, curs_execute_params_batch_doc},
+    {"prepare", (PyCFunction)curs_prepare,
+    METH_VARARGS, curs_prepare_doc},
     {"fetchone", (PyCFunction)curs_fetchone,
      METH_NOARGS, curs_fetchone_doc},
     {"fetchmany", (PyCFunction)curs_fetchmany,

psycopg/pqpath.c

@@ -43,7 +43,7 @@
 #include "psycopg/pgtypes.h"
 #include "psycopg/error.h"
 #include "psycopg/column.h"
-
+#include <stdlib.h>
 #include "psycopg/libpq_support.h"
 #include "libpq-fe.h"
 
@@ -607,6 +607,52 @@ cleanup:
     return rv;
 }
 
+
+/* Get the oid of uint type created by the dolphin plugin, which is only in the pg_catalog by default */
+
+unsigned int
+pq_get_pg_catalog_custom_type_oid(connectionObject *conn, const char *param, PyThreadState **tstate)
+{
+    char query[256];
+    int size;
+    unsigned int rv = 0;
+
+    size = PyOS_snprintf(query, sizeof(query), "select oid from pg_type where typnamespace = 11 and typname= %s", param);
+    if (size < 0 || (size_t)size >= sizeof(query)) {
+        conn_set_error(conn, "query too large");
+        goto cleanup;
+    }
+
+    if (!psyco_green()) {
+        conn_set_result(conn, PQexec(conn->pgconn, query));
+    } else {
+        PyEval_RestoreThread(*tstate);
+        conn_set_result(conn, psyco_exec_green(conn, query));
+        *tstate = PyEval_SaveThread();
+    }
+
+    if (!conn->pgres) {
+        Dprintf("pq_get_pg_catalog_custom_type_oid: PQexec returned NULL");
+        PyEval_RestoreThread(*tstate);
+        if (!PyErr_Occurred()) {
+            conn_set_error(conn, PQerrorMessage(conn->pgconn));
+        }
+        *tstate = PyEval_SaveThread();
+        goto cleanup;
+    }
+    if (PQresultStatus(conn->pgres) != PGRES_TUPLES_OK) {
+        Dprintf("pq_get_pg_catalog_custom_type_oid: result was not TUPLES_OK (%s)",
+                PQresStatus(PQresultStatus(conn->pgres)));
+        goto cleanup;
+    }
+
+    rv = atoi(strdup(PQgetvalue(conn->pgres, 0, 0)));
+    CLEARPGRES(conn->pgres);
+
+cleanup:
+    return rv;
+}
+
 /* Set a session parameter.
  *
  * The function should be called on a locked connection without

psycopg/typecast.c

@@ -459,7 +459,7 @@ PyTypeObject typecastType = {
     0,          /*tp_new*/
 };
 
-static PyObject *
+PyObject *
 typecast_new(PyObject *name, PyObject *values, PyObject *cast, PyObject *base)
 {
     typecastObject *obj;

psycopg/typecast.h

@@ -89,3 +89,4 @@ HIDDEN PyObject *typecast_cast(
     PyObject *self, const char *str, Py_ssize_t len, PyObject *curs);
 
 #endif /* !defined(PSYCOPG_TYPECAST_H) */
+PyObject *typecast_new(PyObject *name, PyObject *values, PyObject *cast, PyObject *base);

psycopg/utils.h

@@ -59,7 +59,7 @@ HIDDEN RAISES BORROWED PyObject *psyco_set_error(
 
 HIDDEN PyObject *psyco_get_decimal_type(void);
 
-HIDDEN PyObject *Bytes_Format(PyObject *format, PyObject *args);
+HIDDEN PyObject *Bytes_Format(PyObject *format, PyObject *args, char place_holder);
 
 
-#endif /* !defined(UTILS_H) */
+#endif /* !defined(UTILS_H) */