输入一条SQL语句,PostgreSQL如何解析输入的SQL,识别SQL类型以及基表/字段等信息?接下来的几节将逐一进行解析. 本节介绍了PostgreSQL的语法分析定义文件gram.y,该文件位于目录src/backend/parser中. 如前所述,PG利用Bison对语法进行分析,Bison输入文件由以下四部分组成: ``` %{ Declarations %} Definitions %% Productions %% User subroutines ``` 本节介绍第一部分Declarations. ### 一、Declarations Declarations与Flex类似,Bison会把这些代码原样拷贝到相应的c文件中(默认为y.tab.c,PG中是gram.c). 名词解释: terminal symbols --> 终结符 non-terminals symbols --> 非终结符 reduce --> 折叠动作,输入为符合集合(终结符/非终结符),输出为匹配该pattern的非终结符 production --> 产生式,比如S -> S E,成为产生式 详细解释请参考《编译原理》. ``` %{ /*#define YYDEBUG 1*/ /*------------------------------------------------------------------------- * * gram.y * POSTGRESQL BISON rules/actions * * Portions Copyright (c) 1996-2018, PostgreSQL Global Development Group * Portions Copyright (c) 1994, Regents of the University of California * * * IDENTIFICATION * src/backend/parser/gram.y * * HISTORY * AUTHOR DATE MAJOR EVENT * Andrew Yu Sept, 1994 POSTQUEL to SQL conversion * Andrew Yu Oct, 1994 lispy code conversion * * NOTES * CAPITALS are used to represent terminal symbols. * non-capitals are used to represent non-terminals. * * In general, nothing in this file should initiate database accesses * nor depend on changeable state (such as SET variables). If you do * database accesses, your code will fail when we have aborted the * current transaction and are just parsing commands to find the next * ROLLBACK or COMMIT. If you make use of SET variables, then you * will do the wrong thing in multi-query strings like this: * SET constraint_exclusion TO off; SELECT * FROM foo; * because the entire string is parsed by gram.y before the SET gets * executed. Anything that depends on the database or changeable state * should be handled during parse analysis so that it happens at the * right time not the wrong time. * * WARNINGS * If you use a list, make sure the datum is a node so that the printing * routines work. * * Sometimes we assign constants to makeStrings. Make sure we don't free * those. * 注意 * 大写字母用于表示终结符号. * 非大写字母用于表示非终结符号. --> 文法中的总结符号和非终结符号 * * 通常来说,这个文件中的逻辑不应启用数据库访问,也不应该依赖于可更改的状态(比如SET变量). * 如果你确实需要数据库访问,业务代码会在回滚当前事务后出错,然后开始解析命令寻找下一个ROLLBACK/COMMIT. * 如果使用了SET变量,那么会在多个查询串中出现错误,比如: * SET constraint_exclusion TO off; SELECT * FROM foo; * 因为整个字符串会在SET执行前被gram.y解析 * 所有依赖数据库或可变状态的事件应该在解析阶段处理以便在正确而非错误的时间发生. * *------------------------------------------------------------------------- */ #include "postgres.h" #include <ctype.h> #include <limits.h> #include "catalog/index.h" #include "catalog/namespace.h" #include "catalog/pg_am.h" #include "catalog/pg_trigger.h" #include "commands/defrem.h" #include "commands/trigger.h" #include "nodes/makefuncs.h" #include "nodes/nodeFuncs.h" #include "parser/gramparse.h" #include "parser/parser.h" #include "parser/parse_expr.h" #include "storage/lmgr.h" #include "utils/date.h" #include "utils/datetime.h" #include "utils/numeric.h" #include "utils/xml.h" /* * Location tracking support --- simpler than bison's default, since we only * want to track the start position not the end position of each nonterminal. * 位置跟踪支持 --- 比bison默认的处理要简单,因为我们只需要跟踪开始位置而非每个非终结符的结束位置. */ #define YYLLOC_DEFAULT(Current, Rhs, N) \ do { \ if ((N) > 0) \ (Current) = (Rhs)[1]; \ else \ (Current) = (-1); \ } while (0) /* * The above macro assigns -1 (unknown) as the parse location of any * nonterminal that was reduced from an empty rule, or whose leftmost * component was reduced from an empty rule. This is problematic * for nonterminals defined like * OptFooList: / * EMPTY * / { ... } | OptFooList Foo { ... } ; * because we'll set -1 as the location during the first reduction and then * copy it during each subsequent reduction, leaving us with -1 for the * location even when the list is not empty. To fix that, do this in the * action for the nonempty rule(s): * if (@$ < 0) @$ = @2; * (Although we have many nonterminals that follow this pattern, we only * bother with fixing @$ like this when the nonterminal's parse location * is actually referenced in some rule.) * 上面的宏将-1(未知数)指定为所有非终结符的解析位置, * 这些非终结符是从空规则折叠(规约)而来的,或者其最左边的组件是从空规则折叠而来. * 对于下面的非终结符,存在问题: * OptFooList: / * EMPTY * / { ... } | OptFooList Foo { ... } ; * 因为在第一次折叠时将设置值为-1,然后在接下来的折叠中拷贝该值, * 这会让就算链表不为空也会一直让位置一直为-1. * 为了修正这一错误,对于非空规则,执行这一动作: * if (@$ < 0) @$ = @2; * * A cleaner answer would be to make YYLLOC_DEFAULT scan all the Rhs * locations until it's found one that's not -1. Then we'd get a correct * location for any nonterminal that isn't entirely empty. But this way * would add overhead to every rule reduction, and so far there's not been * a compelling reason to pay that overhead. * 更清晰的做法是让YYLLOC_DEFAULT扫描所有的Rhs位置直至找到不为-1为止. * 然后我们就可以为完全不为空的非终结符获取正确的位置. * 但这样的做法会增加每个规则折叠的负载,到目前为止,还没有一个令人信服的理由来增加开销. */ /* * Bison doesn't allocate anything that needs to live across parser calls, * so we can easily have it use palloc instead of malloc. This prevents * memory leaks if we error out during parsing. Note this only works with * bison >= 2.0. However, in bison 1.875 the default is to use alloca() * if possible, so there's not really much problem anyhow, at least if * you're building with gcc. * Bison不会在解析器调用期间分配内存,因此我们可以很轻松的使用palloc而不是malloc. * 这可以防止在解析期间出错而导致的内存泄漏.注意这个特性只在2.0+才会起效. * 无论如何,,在bison 1.875这个版本,默认使用alloca分配内存,在使用gcc构建时没有太多问题. */ #define YYMALLOC palloc #define YYFREE pfree /* Private struct for the result of privilege_target production */ //privilege_target产生式结果的私有结构体 typedef struct PrivTarget { GrantTargetType targtype; ObjectType objtype; List *objs; } PrivTarget; /* Private struct for the result of import_qualification production */ //私有结构体 --> import_qualification产生式 typedef struct ImportQual { ImportForeignSchemaType type; List *table_names; } ImportQual; /* ConstraintAttributeSpec yields an integer bitmask of these flags: */ //ConstraintAttributeSpec产生这些标志的整数位掩码 #define CAS_NOT_DEFERRABLE 0x01 #define CAS_DEFERRABLE 0x02 #define CAS_INITIALLY_IMMEDIATE 0x04 #define CAS_INITIALLY_DEFERRED 0x08 #define CAS_NOT_VALID 0x10 #define CAS_NO_INHERIT 0x20 #define parser_yyerror(msg) scanner_yyerror(msg, yyscanner) #define parser_errposition(pos) scanner_errposition(pos, yyscanner) static void base_yyerror(YYLTYPE *yylloc, core_yyscan_t yyscanner, const char *msg); static RawStmt *makeRawStmt(Node *stmt, int stmt_location); static void updateRawStmtEnd(RawStmt *rs, int end_location); static Node *makeColumnRef(char *colname, List *indirection, int location, core_yyscan_t yyscanner); static Node *makeTypeCast(Node *arg, TypeName *typename, int location); static Node *makeStringConst(char *str, int location); static Node *makeStringConstCast(char *str, int location, TypeName *typename); static Node *makeIntConst(int val, int location); static Node *makeFloatConst(char *str, int location); static Node *makeBitStringConst(char *str, int location); static Node *makeNullAConst(int location); static Node *makeAConst(Value *v, int location); static Node *makeBoolAConst(bool state, int location); static RoleSpec *makeRoleSpec(RoleSpecType type, int location); static void check_qualified_name(List *names, core_yyscan_t yyscanner); static List *check_func_name(List *names, core_yyscan_t yyscanner); static List *check_indirection(List *indirection, core_yyscan_t yyscanner); static List *extractArgTypes(List *parameters); static List *extractAggrArgTypes(List *aggrargs); static List *makeOrderedSetArgs(List *directargs, List *orderedargs, core_yyscan_t yyscanner); static void insertSelectOptions(SelectStmt *stmt, List *sortClause, List *lockingClause, Node *limitOffset, Node *limitCount, WithClause *withClause, core_yyscan_t yyscanner); static Node *makeSetOp(SetOperation op, bool all, Node *larg, Node *rarg); static Node *doNegate(Node *n, int location); static void doNegateFloat(Value *v); static Node *makeAndExpr(Node *lexpr, Node *rexpr, int location); static Node *makeOrExpr(Node *lexpr, Node *rexpr, int location); static Node *makeNotExpr(Node *expr, int location); static Node *makeAArrayExpr(List *elements, int location); static Node *makeSQLValueFunction(SQLValueFunctionOp op, int32 typmod, int location); static Node *makeXmlExpr(XmlExprOp op, char *name, List *named_args, List *args, int location); static List *mergeTableFuncParameters(List *func_args, List *columns); static TypeName *TableFuncTypeName(List *columns); static RangeVar *makeRangeVarFromAnyName(List *names, int position, core_yyscan_t yyscanner); static void SplitColQualList(List *qualList, List **constraintList, CollateClause **collClause, core_yyscan_t yyscanner); static void processCASbits(int cas_bits, int location, const char *constrType, bool *deferrable, bool *initdeferred, bool *not_valid, bool *no_inherit, core_yyscan_t yyscanner); static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query); %} ``` ### 二、参考资料 [Flex&Bison](http://web.iitd.ac.in/~sumeet/flex__bison.pdf)