PostgreSQL 14在MVCC上面有所增强，以保证在大批量客户端连接场景下性能不会出现显著下降（详细可参见参考资料中的链接）。本节介绍了提交记录“Don't compute global horizons while building snapshots.”中procarray.c中的修改以及涉及的相关数据结构。 ### 一、数据结构 N/A ### 二、源码解读 回顾这次代码commit的说明,会有助于理解committer为什么这样修改: ``` To make GetSnapshotData() more scalable, it cannot not look at at each proc's xmin: While snapshot contents do not need to change whenever a read-only transaction commits or a snapshot is released, a proc's xmin is modified in those cases. The frequency of xmin modifications leads to, particularly on higher core count systems, many cache misses inside GetSnapshotData(), despite the data underlying a snapshot not changing. That is the most significant source of GetSnapshotData() scaling poorly on larger systems. GetSnapshotData的性能随着连接的增长会出现明显的下降,其根源是xmin的频繁更新导致该函数 在执行过程中出现许多缓存没有命中的情况,导致效率低下. 对于只读事务或者快照释放xmin无需更新的情况,内核也更新了xmin. Without accessing xmins, GetSnapshotData() cannot calculate accurate horizons / thresholds as it has so far. But we don't really have to: The horizons don't actually change that much between GetSnapshotData() calls. Nor are the horizons actually used every time a snapshot is built. 如果不访问PROC的xmins,GetSnapshotData不能计算精确的基准线/阈值. 但实际上在GetSnapshotData调用时并不需要更新基准线,在快照建立时也不需要使用基准线. The trick this commit introduces is to delay computation of accurate horizons until there use and using horizon boundaries to determine whether accurate horizons need to be computed. 因此,在需要的时候才计算精确的基准线. The use of RecentGlobal[Data]Xmin to decide whether a row version could be removed has been replaces with new GlobalVisTest* functions. These use two thresholds to determine whether a row can be pruned: 1) definitely_needed, indicating that rows deleted by XIDs >= definitely_needed are definitely still visible. 2) maybe_needed, indicating that rows deleted by XIDs < maybe_needed can definitely be removed GetSnapshotData() updates definitely_needed to be the xmin of the computed snapshot. When testing whether a row can be removed (with GlobalVisTestIsRemovableXid()) and the tested XID falls in between the two (i.e. XID >= maybe_needed && XID < definitely_needed) the boundaries can be recomputed to be more accurate. As it is not cheap to compute accurate boundaries, we limit the number of times that happens in short succession. As the boundaries used by GlobalVisTestIsRemovableXid() are never reset (with maybe_needed updated by GetSnapshotData()), it is likely that further test can benefit from an earlier computation of accurate horizons. To avoid regressing performance when old_snapshot_threshold is set (as that requires an accurate horizon to be computed), heap_page_prune_opt() doesn't unconditionally call TransactionIdLimitedForOldSnapshots() anymore. Both the computation of the limited horizon, and the triggering of errors (with SetOldSnapshotThresholdTimestamp()) is now only done when necessary to remove tuples. This commit just removes the accesses to PGXACT->xmin from GetSnapshotData(), but other members of PGXACT residing in the same cache line are accessed. Therefore this in itself does not result in a significant improvement. Subsequent commits will take advantage of the fact that GetSnapshotData() now does not need to access xmins anymore. 本次提交只是移除了GetSnapshotData函数对PGXACT->xmin的访问,因此本次提交并没有带来明显的性能提升. Note: This contains a workaround in heap_page_prune_opt() to keep the snapshot_too_old tests working. While that workaround is ugly, the tests currently are not meaningful, and it seems best to address them separately. ``` **procarray.c(第二部分)** ```diff --- a/src/backend/storage/ipc/procarray.c +++ b/src/backend/storage/ipc/procarray.c /* @@ -1544,12 +1841,9 @@ GetMaxSnapshotSubxidCount(void) * current transaction (this is the same as MyPgXact->xmin). * RecentXmin: the xmin computed for the most recent snapshot. XIDs * older than this are known not running any more. - * RecentGlobalXmin: the global xmin (oldest TransactionXmin across all - * running transactions, except those running LAZY VACUUM). This is - * the same computation done by - * GetOldestXmin(NULL, PROCARRAY_FLAGS_VACUUM). - * RecentGlobalDataXmin: the global xmin for non-catalog tables - * >= RecentGlobalXmin + * + * And try to advance the bounds of GlobalVisSharedRels, GlobalVisCatalogRels, + * GlobalVisDataRels for the benefit of theGlobalVisTest* family of functions. * * Note: this function should probably not be called with an argument that's * not statically allocated (see xip allocation below). /* GetSnapshotData : 返回运行中的事务信息,xmin:xip:xmax */ @@ -1560,12 +1854,12 @@ GetSnapshotData(Snapshot snapshot) ProcArrayStruct *arrayP = procArray;//proc数组 TransactionId xmin; TransactionId xmax; //删掉globalxmin - TransactionId globalxmin; int index; int count = 0; int subcount = 0; bool suboverflowed = false; FullTransactionId latest_completed; //增加最旧的xid + TransactionId oldestxid; TransactionId replication_slot_xmin = InvalidTransactionId; TransactionId replication_slot_catalog_xmin = InvalidTransactionId; @@ -1610,13 +1904,15 @@ GetSnapshotData(Snapshot snapshot) //---------- 以共享方式获取ProcArrayLock锁,遍历Procs数组期间不允许其他进程修改Procs数组 LWLockAcquire(ProcArrayLock, LW_SHARED); latest_completed = ShmemVariableCache->latestCompletedXid; //从共享变量中获取oldestxid + oldestxid = ShmemVariableCache->oldestXid; + /* xmax is always latestCompletedXid + 1 */ /* xmax = latestCompletedXid + 1 */ xmax = XidFromFullTransactionId(latest_completed); TransactionIdAdvance(xmax); Assert(TransactionIdIsNormal(xmax)); /* initialize xmin calculation with xmax */ //初始化xmin/xmax,删掉globalxmin - globalxmin = xmin = xmax; + xmin = xmax; snapshot->takenDuringRecovery = RecoveryInProgress(); @@ -1645,12 +1941,6 @@ GetSnapshotData(Snapshot snapshot) (PROC_IN_LOGICAL_DECODING | PROC_IN_VACUUM)) continue; /* 不再维护globalxmin */ - /* Update globalxmin to be the smallest valid xmin */ - xid = UINT32_ACCESS_ONCE(pgxact->xmin); - if (TransactionIdIsNormal(xid) && - NormalTransactionIdPrecedes(xid, globalxmin)) - globalxmin = xid; - /* Fetch xid just once - see GetNewTransactionId */ xid = UINT32_ACCESS_ONCE(pgxact->xid); @@ -1766,34 +2056,78 @@ GetSnapshotData(Snapshot snapshot) //------------ 释放ProcArrayLock锁 LWLockRelease(ProcArrayLock); - /* - * Update globalxmin to include actual process xids. This is a slightly - * different way of computing it than GetOldestXmin uses, but should give - * the same result. - */ - if (TransactionIdPrecedes(xmin, globalxmin)) - globalxmin = xmin; /* 维护GlobalVis相关的状态变量 注意 : 前面有一段代码对xmin进行了处理, for (size_t pgxactoff = 0; pgxactoff < numProcs; pgxactoff++) 遍历ProcArray数组中所有后台进程的的事务,获取xmin */ + /* maintain state for GlobalVis* */ + { + TransactionId def_vis_xid; + TransactionId def_vis_xid_data; + FullTransactionId def_vis_fxid; + FullTransactionId def_vis_fxid_data; + FullTransactionId oldestfxid; - /* Update global variables too */ - RecentGlobalXmin = globalxmin - vacuum_defer_cleanup_age; - if (!TransactionIdIsNormal(RecentGlobalXmin)) - RecentGlobalXmin = FirstNormalTransactionId; + /* + * Converting oldestXid is only safe when xid horizon cannot advance, + * i.e. holding locks. While we don't hold the lock anymore, all the + * necessary data has been gathered with lock held. + */ //转换为full XID + oldestfxid = FullXidRelativeTo(latest_completed, oldestxid); - /* Check whether there's a replication slot requiring an older xmin. */ - if (TransactionIdIsValid(replication_slot_xmin) && - NormalTransactionIdPrecedes(replication_slot_xmin, RecentGlobalXmin)) - RecentGlobalXmin = replication_slot_xmin; + /* apply vacuum_defer_cleanup_age */ //避免回卷对xmin进行处理 + def_vis_xid_data = + TransactionIdRetreatedBy(xmin, vacuum_defer_cleanup_age); - /* Non-catalog tables can be vacuumed if older than this xid */ - RecentGlobalDataXmin = RecentGlobalXmin; + /* Check whether there's a replication slot requiring an older xmin. */ //在def_vis_xid_data和replication_slot_xmin之间取老旧的那个 + def_vis_xid_data = + TransactionIdOlder(def_vis_xid_data, replication_slot_xmin); - /* - * Check whether there's a replication slot requiring an older catalog - * xmin. - */ - if (TransactionIdIsNormal(replication_slot_catalog_xmin) && - NormalTransactionIdPrecedes(replication_slot_catalog_xmin, RecentGlobalXmin)) - RecentGlobalXmin = replication_slot_catalog_xmin; + /* + * Rows in non-shared, non-catalog tables possibly could be vacuumed + * if older than this xid. + */ /* def_vis_xid赋值,初始化为def_vis_xid_data */ + def_vis_xid = def_vis_xid_data; + + /* + * Check whether there's a replication slot requiring an older catalog + * xmin. + */ /* 与replication_slot_catalog_xmin比较,取较旧的那个 */ + def_vis_xid = + TransactionIdOlder(replication_slot_catalog_xmin, def_vis_xid); + /* def_vis_xid转换为以latest_completed为基数的full xid def_vis_fxid_data转换为以latest_completed为基数的full xid */ + def_vis_fxid = FullXidRelativeTo(latest_completed, def_vis_xid); + def_vis_fxid_data = FullXidRelativeTo(latest_completed, def_vis_xid_data); + + /* + * Check if we can increase upper bound. As a previous + * GlobalVisUpdate() might have computed more aggressive values, don't + * overwrite them if so. + */ /* 检查是否可以增加上界. 如果先前通过函数GlobalVisUpdate计算已得到了更为激进的值,那么不应覆盖该值. */ + GlobalVisSharedRels.definitely_needed = + FullTransactionIdNewer(def_vis_fxid, + GlobalVisSharedRels.definitely_needed); + GlobalVisCatalogRels.definitely_needed = + FullTransactionIdNewer(def_vis_fxid, + GlobalVisCatalogRels.definitely_needed); + GlobalVisDataRels.definitely_needed = + FullTransactionIdNewer(def_vis_fxid_data, + GlobalVisDataRels.definitely_needed); + + /* + * Check if we know that we can initialize or increase the lower + * bound. Currently the only cheap way to do so is to use + * ShmemVariableCache->oldestXid as input. + * + * We should definitely be able to do better. We could e.g. put a + * global lower bound value into ShmemVariableCache. + */ /* 更新maybe_needed,取maybe_needed和oldestfxid之间较新的那个 */ + GlobalVisSharedRels.maybe_needed = + FullTransactionIdNewer(GlobalVisSharedRels.maybe_needed, + oldestfxid); + GlobalVisCatalogRels.maybe_needed = + FullTransactionIdNewer(GlobalVisCatalogRels.maybe_needed, + oldestfxid); + GlobalVisDataRels.maybe_needed = + FullTransactionIdNewer(GlobalVisDataRels.maybe_needed, + oldestfxid); + } RecentXmin = xmin; @@ -3291,6 +3625,255 @@ DisplayXidCache(void) } #endif /* XIDCACHE_DEBUG */ +/* + * If rel != NULL, return test state appropriate for relation, otherwise + * return state usable for all relations. The latter may consider XIDs as + * not-yet-visible-to-everyone that a state for a specific relation would + * already consider visible-to-everyone. + * + * This needs to be called while a snapshot is active or registered, otherwise + * there are wraparound and other dangers. + * + * See comment for GlobalVisState for details. + */ /* 计算对应该rel的状态信息,如rel为NULL则返回对所有rel适用的状态信息. 对于GlobalVisState中认为还没有可见的XIDs,在后续会修改为对所有事务可见. 在快照活动或者注册时调用该函数,否则存在回卷和其他危险. */ +GlobalVisState * +GlobalVisTestFor(Relation rel) +{ + bool need_shared;//全局共享表 + bool need_catalog;//字典表 + GlobalVisState *state;//返回值 + + /* XXX: we should assert that a snapshot is pushed or registered */ + Assert(RecentXmin);//快照已入栈或注册 + + if (!rel) + need_shared = need_catalog = true;//rel为NULL,返回通用版 + else + { + /* + * Other kinds currently don't contain xids, nor always the necessary + * logical decoding markers. + */ + Assert(rel->rd_rel->relkind == RELKIND_RELATION || + rel->rd_rel->relkind == RELKIND_MATVIEW || + rel->rd_rel->relkind == RELKIND_TOASTVALUE);//数据表或者物化视图或者TOAST值 + //数据表全局共享或者在恢复过程中 + need_shared = rel->rd_rel->relisshared || RecoveryInProgress(); //字典表或者在逻辑解码中访问 + need_catalog = IsCatalogRelation(rel) || RelationIsAccessibleInLogicalDecoding(rel); + } + //返回相应的全局变量 //这些变量会在GetSnapshotData函数中更新 + if (need_shared) + state = &GlobalVisSharedRels; + else if (need_catalog) + state = &GlobalVisCatalogRels; + else + state = &GlobalVisDataRels; + + Assert(FullTransactionIdIsValid(state->definitely_needed) && + FullTransactionIdIsValid(state->maybe_needed)); + + return state; +} + +/* + * Return true if it's worth updating the accurate maybe_needed boundary. + * + * As it is somewhat expensive to determine xmin horizons, we don't want to + * repeatedly do so when there is a low likelihood of it being beneficial. + * + * The current heuristic is that we update only if RecentXmin has changed + * since the last update. If the oldest currently running transaction has not + * finished, it is unlikely that recomputing the horizon would be useful. /* 如果值得更新maybe_needed,则返回true. 由于确定xmin边界比较昂贵,因此在益处不大时我们不希望重复执行该操作. 在最后一次更新后,RecentXmin出现变更时 */ + */ +static bool +GlobalVisTestShouldUpdate(GlobalVisState *state) +{ + /* hasn't been updated yet */ /* ComputeXidHorizonsResultLastXmin还没有更新 */ + if (!TransactionIdIsValid(ComputeXidHorizonsResultLastXmin)) + return true; + + /* + * If the maybe_needed/definitely_needed boundaries are the same, it's + * unlikely to be beneficial to refresh boundaries. + */ /* maybe_needed >= definitely_needed,不需要更新 */ + if (FullTransactionIdFollowsOrEquals(state->maybe_needed, + state->definitely_needed)) + return false; + + /* does the last snapshot built have a different xmin? */ /* 否则,如果最后一个快照使用不同的xmin构建,则返回true */ + return RecentXmin != ComputeXidHorizonsResultLastXmin; +} + +static void +GlobalVisUpdateApply(ComputeXidHorizonsResult *horizons) +{ /* 更新全局数据结构GlobalVisSharedRels/GlobalVisCatalogRels/GlobalVisDataRels */ + GlobalVisSharedRels.maybe_needed = + FullXidRelativeTo(horizons->latest_completed, + horizons->shared_oldest_nonremovable); + GlobalVisCatalogRels.maybe_needed = + FullXidRelativeTo(horizons->latest_completed, + horizons->catalog_oldest_nonremovable); + GlobalVisDataRels.maybe_needed = + FullXidRelativeTo(horizons->latest_completed, + horizons->data_oldest_nonremovable); + + /* + * In longer running transactions it's possible that transactions we + * previously needed to treat as running aren't around anymore. So update + * definitely_needed to not be earlier than maybe_needed. + */ + GlobalVisSharedRels.definitely_needed = + FullTransactionIdNewer(GlobalVisSharedRels.maybe_needed, + GlobalVisSharedRels.definitely_needed); + GlobalVisCatalogRels.definitely_needed = + FullTransactionIdNewer(GlobalVisCatalogRels.maybe_needed, + GlobalVisCatalogRels.definitely_needed); + GlobalVisDataRels.definitely_needed = + FullTransactionIdNewer(GlobalVisDataRels.maybe_needed, + GlobalVisDataRels.definitely_needed); + + ComputeXidHorizonsResultLastXmin = RecentXmin; +} + +/* + * Update boundaries in GlobalVis{Shared,Catalog, Data}Rels + * using ComputeXidHorizons(). + */ /* 使用ComputeXidHorizons更新GlobalVis{Shared,Catalog, Data}Rels结构体 */ +static void +GlobalVisUpdate(void) +{ + ComputeXidHorizonsResult horizons; + + /* updates the horizons as a side-effect */ + ComputeXidHorizons(&horizons); +} + +/* + * Return true if no snapshot still considers fxid to be running. + * + * The state passed needs to have been initialized for the relation fxid is + * from (NULL is also OK), otherwise the result may not be correct. + * + * See comment for GlobalVisState for details. + */ /* 如已没有snapshot认为fxid正在运行(即均可见),则返回true */ +bool +GlobalVisTestIsRemovableFullXid(GlobalVisState *state, + FullTransactionId fxid) +{ + /* + * If fxid is older than maybe_needed bound, it definitely is visible to + * everyone. + */ // 比maybe_needed要小,则一定可见 + if (FullTransactionIdPrecedes(fxid, state->maybe_needed)) + return true; + + /* + * If fxid is >= definitely_needed bound, it is very likely to still be + * considered running. + */ // 比definitely_needed大,则一定不可见(正在运行) + if (FullTransactionIdFollowsOrEquals(fxid, state->definitely_needed)) + return false; + + /* + * fxid is between maybe_needed and definitely_needed, i.e. there might or + * might not exist a snapshot considering fxid running. If it makes sense, + * update boundaries and recheck. + */ /* fxid ∈ [maybe_needed,definitely_needed],则调用GlobalVisUpdate(间接调用ComputeXidHorizons)更新 */ + if (GlobalVisTestShouldUpdate(state)) + { + GlobalVisUpdate(); + + Assert(FullTransactionIdPrecedes(fxid, state->definitely_needed)); + + return FullTransactionIdPrecedes(fxid, state->maybe_needed); + } + else + return false; +} + +/* + * Wrapper around GlobalVisTestIsRemovableFullXid() for 32bit xids. + * + * It is crucial that this only gets called for xids from a source that + * protects against xid wraparounds (e.g. from a table and thus protected by + * relfrozenxid). + */ /* 适配32bit的xid */ +bool +GlobalVisTestIsRemovableXid(GlobalVisState *state, TransactionId xid) +{ + FullTransactionId fxid; + + /* + * Convert 32 bit argument to FullTransactionId. We can do so safely + * because we know the xid has to, at the very least, be between + * [oldestXid, nextFullXid), i.e. within 2 billion of xid. To avoid taking + * a lock to determine either, we can just compare with + * state->definitely_needed, which was based on those value at the time + * the current snapshot was built. + */ + fxid = FullXidRelativeTo(state->definitely_needed, xid); + + return GlobalVisTestIsRemovableFullXid(state, fxid); +} + +/* + * Return FullTransactionId below which all transactions are not considered + * running anymore. + * + * Note: This is less efficient than testing with + * GlobalVisTestIsRemovableFullXid as it likely requires building an accurate + * cutoff, even in the case all the XIDs compared with the cutoff are outside + * [maybe_needed, definitely_needed). + */ /* 返回maybe_needed,小于该值的,必可见 */ +FullTransactionId +GlobalVisTestNonRemovableFullHorizon(GlobalVisState *state) +{ + /* acquire accurate horizon if not already done */ + if (GlobalVisTestShouldUpdate(state)) + GlobalVisUpdate(); + + return state->maybe_needed; +} + +/* Convenience wrapper around GlobalVisTestNonRemovableFullHorizon */ /* 上面函数的32bit xid版本 */ +TransactionId +GlobalVisTestNonRemovableHorizon(GlobalVisState *state) +{ + FullTransactionId cutoff; + + cutoff = GlobalVisTestNonRemovableFullHorizon(state); + + return XidFromFullTransactionId(cutoff); +} + +/* + * Convenience wrapper around GlobalVisTestFor() and + * GlobalVisTestIsRemovableFullXid(), see their comments. + */ /* 下面两个函数只是简化函数调用 */ +bool +GlobalVisIsRemovableFullXid(Relation rel, FullTransactionId fxid) +{ + GlobalVisState *state; + + state = GlobalVisTestFor(rel); + + return GlobalVisTestIsRemovableFullXid(state, fxid); +} + +/* + * Convenience wrapper around GlobalVisTestFor() and + * GlobalVisTestIsRemovableXid(), see their comments. + */ +bool +GlobalVisCheckRemovableXid(Relation rel, TransactionId xid) +{ + GlobalVisState *state; + + state = GlobalVisTestFor(rel); + + return GlobalVisTestIsRemovableXid(state, xid); +} + /* * Convert a 32 bit transaction id into 64 bit transaction id, by assuming it * is within MaxTransactionId / 2 of XidFromFullTransactionId(rel). ``` ### 三、跟踪分析 N/A ### 四、参考资料 1.[Improving connection scalability: GetSnapshotData()](https://www.postgresql.org/message-id/20200301083601.ews6hz5dduc3w2se%40alap3.anarazel.de) 2.[snapshot scalability: Don't compute global horizons while building snapshots.](https://git.postgresql.org/gitweb/?p=postgresql.git;a=commitdiff;h=dc7420c2c9274a283779ec19718d2d16323640c0)