PostgreSQL 14在MVCC上面有所增强，以保证在大批量客户端连接场景下性能不会出现显著下降（详细可参见参考资料中的链接）。本节介绍了提交记录“Don't compute global horizons while building snapshots.”中heapam.c中的修改以及涉及的相关数据结构。 ### 一、数据结构 **GlobalVisState** ```C /* * State for the GlobalVisTest* family of functions. Those functions can * e.g. be used to decide if a deleted row can be removed without violating * MVCC semantics: If the deleted row's xmax is not considered to be running * by anyone, the row can be removed. * 用于配合GlobalVisTest*函数族使用的数据结构。 * 这些函数用于确定删除的行在不违反MVCC语义的情况下是否可以被清除： * 如果被删除的行的xmax与所有运行的事务无关，那么可以清除。 * * To avoid slowing down GetSnapshotData(), we don't calculate a precise * cutoff XID while building a snapshot (looking at the frequently changing * xmins scales badly). Instead we compute two boundaries while building the * snapshot: * * 1) definitely_needed, indicating that rows deleted by XIDs >= * definitely_needed are definitely still visible. * * 2) maybe_needed, indicating that rows deleted by XIDs < maybe_needed can * definitely be removed * * 为了避免影响GetSnapshotData()的性能，在构建快照时我们不会计算精确的cutoff XID， * 相反，我们会在构建快照时计算两个边界，分别是： * 1）definitely_needed，表示如果删除事务的XIDs >= definitely_needed，则可见 * 2）maybe_needed，表示被XIDs < maybe_needed 事务删除的，可以确定被清除 * * When testing an XID that falls in between the two (i.e. XID >= maybe_needed * && XID < definitely_needed), the boundaries can be recomputed (using * ComputeXidHorizons()) to get a more accurate answer. This is cheaper than * maintaining an accurate value all the time. * 对于XID >= maybe_needed && XID < definitely_needed的XID， * 边界需要使用函数ComputeXidHorizons重新计算来获取更精确的结果，这比在所有情况下都维护一个精确值成本要低 * * As it is not cheap to compute accurate boundaries, we limit the number of * times that happens in short succession. See GlobalVisTestShouldUpdate(). * 计算精确的边界成本高昂，因此需要限制短时间内出现的次数，详细参见GlobalVisTestShouldUpdate函数 * * There are three backend lifetime instances of this struct, optimized for * different types of relations. As e.g. a normal user defined table in one * database is inaccessible to backends connected to another database, a test * specific to a relation can be more aggressive than a test for a shared * relation. Currently we track four different states: * 对于不同的关系，出于优化的目的，该结构有三种不同的backend生命周期。 * 比如某个数据库中用户定义的普通表对于连接到另外一个数据库的backends是不可见的， * 对某个特定关系的验证比起共享关系的验证要更为积极。我们跟踪四种不同的状态： * * 1) GlobalVisSharedRels, which only considers an XID's * effects visible-to-everyone if neither snapshots in any database, nor a * replication slot's xmin, nor a replication slot's catalog_xmin might * still consider XID as running. * 1）GlobalVisSharedRels，事务对该Relation的操作会影响所有数据库中的快照、复制槽的xmin和复制槽的catalog_xmin。 * * 2) GlobalVisCatalogRels, which only considers an XID's * effects visible-to-everyone if neither snapshots in the current * database, nor a replication slot's xmin, nor a replication slot's * catalog_xmin might still consider XID as running. * * I.e. the difference to GlobalVisSharedRels is that * snapshot in other databases are ignored. * 2）GlobalVisCatalogRels，事务对该Relation的操作会影响当前数据库中的快照、复制槽的xmin和复制槽的catalog_xmin * 与GlobalVisSharedRels不同之处是其他数据库的快照会被忽略 * * 3) GlobalVisDataRels, which only considers an XID's * effects visible-to-everyone if neither snapshots in the current * database, nor a replication slot's xmin consider XID as running. * * I.e. the difference to GlobalVisCatalogRels is that * replication slot's catalog_xmin is not taken into account. * * 3）GlobalVisDataRels，事务对该Relation的操作会影响当前数据库中的快照、复制槽的xmin * 与GlobalVisCatalogRels不同的是catalog_xmin不作考虑 * * 4) GlobalVisTempRels, which only considers the current session, as temp * tables are not visible to other sessions. * * 4）GlobalVisTempRels，只考虑当前会话，临时表对其他session不可见 * * GlobalVisTestFor(relation) returns the appropriate state * for the relation. * GlobalVisTestFor(relation)返回关系正确的状态 * * The boundaries are FullTransactionIds instead of TransactionIds to avoid * wraparound dangers. There e.g. would otherwise exist no procarray state to * prevent maybe_needed to become old enough after the GetSnapshotData() * call. * 边界使用FullTransactionIds而不是TransactionIds以避免因为回卷而导致的错误。 * 例如，在GetSnapshotData()调用后，将不存在procarray状态来防止maybe_needed变得32bit XID中存在的过老现象。 * * The typedef is in the header. */ struct GlobalVisState { /* XIDs >= are considered running by some backend */ //大于等于该值的XIDs：其他活动backend可见 FullTransactionId definitely_needed; /* XIDs < are not considered to be running by any backend */ //小于该值的XIDs：活动backend不可见 FullTransactionId maybe_needed; }; /* * A 64 bit value that contains an epoch and a TransactionId. This is * wrapped in a struct to prevent implicit conversion to/from TransactionId. * Not all values represent valid normal XIDs. * 号外：PG 14已支持64 bit XID，令人厌烦的XID回卷问题不复存在 */ typedef struct FullTransactionId { uint64 value; } FullTransactionId; ``` ### 二、源码解读 **diff heapam.c** 修改heapam.c，目的是去掉RecentGlobalXmin的访问，改为通过GlobalVisXXX配合GlobalVisState Struct实现类似的功能 ```Diff --- a/src/backend/access/heap/heapam.c +++ b/src/backend/access/heap/heapam.c @@ -1517,6 +1517,7 @@ heap_hot_search_buffer(ItemPointer tid, Relation relation, Buffer buffer, bool at_chain_start; bool valid; bool skip; + GlobalVisState *vistest = NULL;//新引入的数据结构，全局访问状态，详见GlobalVisState Struct /* If this is not the first call, previous call returned a (live!) tuple */ if (all_dead) @@ -1527,7 +1528,8 @@ heap_hot_search_buffer(ItemPointer tid, Relation relation, Buffer buffer, at_chain_start = first_call; skip = !first_call; - Assert(TransactionIdIsValid(RecentGlobalXmin)); + /* XXX: we should assert that a snapshot is pushed or registered */ + Assert(TransactionIdIsValid(RecentXmin));//去掉全局变量RecentGlobalXmin的访问，转而确认快照入栈和注册 Assert(BufferGetBlockNumber(buffer) == blkno); /* Scan through possible multiple members of HOT-chain */ @@ -1616,9 +1618,14 @@ heap_hot_search_buffer(ItemPointer tid, Relation relation, Buffer buffer, * Note: if you change the criterion here for what is "dead", fix the * planner's get_actual_variable_range() function to match. */ - if (all_dead && *all_dead && - !HeapTupleIsSurelyDead(heapTuple, RecentGlobalXmin)) - *all_dead = false; + if (all_dead && *all_dead)//去掉与RecentGlobalXmin相关的代码，改为从全局访问状态结构体中获取信息 + { + if (!vistest) + vistest = GlobalVisTestFor(relation);//在后续介绍 + + if (!HeapTupleIsSurelyDead(heapTuple, vistest)) + *all_dead = false; + } /* * Check to see if HOT chain continues past this tuple; if so fetch ``` **GlobalVisTestFor** ```C /* * If rel != NULL, return test state appropriate for relation, otherwise * return state usable for all relations. The latter may consider XIDs as * not-yet-visible-to-everyone that a state for a specific relation would * already consider visible-to-everyone. * 如果rel有效（不为NULL），返回对应该关系的验证状态，否则返回对所有关系均可用的状态。 * 后续可能会认为XIDs为对所有仍未可见的状态，该状态对于某个指定的关系可能已经认为对所有事务可见。 * * This needs to be called while a snapshot is active or registered, otherwise * there are wraparound and other dangers. * 该例程在快照活动或已注册时调用，否则存在回卷或其他风险。 * * See comment for GlobalVisState for details. * 详细可参见GlobalVisState */ GlobalVisState * GlobalVisTestFor(Relation rel) { bool need_shared;//共享？ bool need_catalog;//数据字典？ GlobalVisState *state;//访问状态 /* XXX: we should assert that a snapshot is pushed or registered */ //快照已入栈或注册 Assert(RecentXmin); if (!rel) need_shared = need_catalog = true;//rel为NULL，则均值为true else { /* * Other kinds currently don't contain xids, nor always the necessary * logical decoding markers. * 还没有包含xids的其他类型，或者通常是逻辑解码标记器 */ Assert(rel->rd_rel->relkind == RELKIND_RELATION || rel->rd_rel->relkind == RELKIND_MATVIEW || rel->rd_rel->relkind == RELKIND_TOASTVALUE); need_shared = rel->rd_rel->relisshared || RecoveryInProgress();//共享的关系（跨数据库）或者正处于恢复过程 need_catalog = IsCatalogRelation(rel) || RelationIsAccessibleInLogicalDecoding(rel);//数据字典表或者逻辑解码可访问的关系 } if (need_shared) state = &GlobalVisSharedRels;//全局共享 else if (need_catalog) state = &GlobalVisCatalogRels;//当前数据库中的Catalog else if (RELATION_IS_LOCAL(rel)) state = &GlobalVisTempRels;//临时表 else state = &GlobalVisDataRels;//常规表 Assert(FullTransactionIdIsValid(state->definitely_needed) && FullTransactionIdIsValid(state->maybe_needed)); return state; } ``` **** ``` ``` ### 三、跟踪分析 N/A ### 四、参考资料 1.[Improving connection scalability: GetSnapshotData()](https://www.postgresql.org/message-id/20200301083601.ews6hz5dduc3w2se%40alap3.anarazel.de) 2.[snapshot scalability: Don't compute global horizons while building snapshots.](https://git.postgresql.org/gitweb/?p=postgresql.git;a=commitdiff;h=dc7420c2c9274a283779ec19718d2d16323640c0)