<?xml version="1.0" encoding="utf-8"?><feed xmlns="http://www.w3.org/2005/Atom" xml:lang="en"><generator uri="https://jekyllrb.com/" version="4.3.4">Jekyll</generator><link href="https://blog.changcodes.com/feed/zh.xml" rel="self" type="application/atom+xml" /><link href="https://blog.changcodes.com/" rel="alternate" type="text/html" hreflang="en" /><updated>2026-10-25T15:03:32+00:00</updated><id>https://blog.changcodes.com/feed/zh.xml</id><title type="html">Chang Codes | Zh</title><subtitle>Engineering notes on workflows, data integrity and failure safety.</subtitle><entry xml:lang="zh-CN"><title type="html">API 请求处理中的失败安全：限流、开关、鉴权与可观测性的默认值设计</title><link href="https://blog.changcodes.com/zh/posts/api-failure-safety/" rel="alternate" type="text/html" title="API 请求处理中的失败安全：限流、开关、鉴权与可观测性的默认值设计" /><published>2026-07-19T00:00:00+00:00</published><updated>2026-07-19T00:00:00+00:00</updated><id>https://blog.changcodes.com/zh/posts/api-failure-safety</id><content type="html" xml:base="https://blog.changcodes.com/zh/posts/api-failure-safety/"><![CDATA[<h2 id="1-问题与目标">1. 问题与目标</h2>

<p>我们的后端服务同时承载几类流量：</p>

<ul>
  <li>面向用户的自助接口：提交申请、查询状态、查看我的记录；</li>
  <li>外部系统的回调；</li>
  <li>运维接口；</li>
  <li>一个单页应用（SPA）前端。</li>
</ul>

<p>部分用户请求会触发耗时较长、成本较高的外部操作，例如创建 Merge Request 和运行流水线。</p>

<p>在这样的服务里，横切关注点（限流、功能开关、鉴权、幂等、指标）本身也会出问题：配置缺失、配置写错、依赖服务异常、埋点代码抛出异常。我们的目标是：</p>

<ul>
  <li>为每个横切关注点明确定义它在”依赖缺失、配置错误、依赖失败”时的行为；</li>
  <li>配置错误不会开放本应关闭的功能，也不会拦截本应放行的流量；</li>
  <li>指标采集永远不改变业务结果；</li>
  <li>错误响应对客户端有用，同时不泄露内部细节。</li>
</ul>

<h2 id="2-约束与考量">2. 约束与考量</h2>

<ul>
  <li><strong>服务多实例部署</strong>，没有专门用于限流的共享缓存。</li>
  <li><strong>配置来自配置中心并支持热更新。</strong> 这意味着配置可能缺失、为空或拼写错误，而且会在运行中发生变化。</li>
  <li><strong>调用方身份来自统一登录 SDK，字段形态不统一。</strong> 有时是邮箱形式的账号，有时是工号，需要统一解析成同一个身份。</li>
  <li><strong>不修改既有接口的返回结构。</strong> 部分历史接口直接返回 <code class="language-plaintext highlighter-rouge">Map</code>，横切逻辑不能依赖统一的返回类型。</li>
  <li><strong>前端发布后存在旧页面。</strong> 已打开的旧页面会请求新版本中已被删除的带 hash 静态资源。</li>
  <li><strong>团队约定可观测性代码只读不写。</strong> 埋点只能读取业务已经解析出的值，不能反过来影响业务逻辑。</li>
</ul>

<h2 id="3-方案">3. 方案</h2>

<h3 id="31-拦截器的职责划分与顺序">3.1 拦截器的职责划分与顺序</h3>

<p>拦截器按以下顺序注册，Spring 按注册顺序执行：</p>

<ol>
  <li><strong>限流</strong>：仅作用于标注了 <code class="language-plaintext highlighter-rouge">@RateLimit</code> 的处理方法。</li>
  <li><strong>特定路径的鉴权。</strong></li>
  <li><strong>API 延迟与状态码指标。</strong></li>
  <li><strong>业务活动指标</strong>：只作用于一个明确列出的路径白名单。</li>
</ol>

<p>限流排在第一位有两个原因。其一，被拒绝的请求不应消耗任何后续处理。其二，业务活动计数衡量的是”用户完成了多少次业务动作”，被 429 拒绝的请求不应计入其中；它们由单独的”被限流次数”指标统计。</p>

<p>业务活动指标使用路径白名单而不是通配符，这样与该业务无关的流量根本不会进入这个拦截器。</p>

<p>限流和鉴权都放在拦截器里实现，在参数绑定之前执行，有两个好处：</p>

<ul>
  <li>格式错误的请求体无法先触发 400、绕开或抢在鉴权之前返回；</li>
  <li>拦截器自己写出响应，不需要修改各个接口的返回类型。</li>
</ul>

<h3 id="32-注解驱动在参数绑定前执行的限流">3.2 注解驱动、在参数绑定前执行的限流</h3>

<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nd">@PostMapping</span><span class="o">(</span><span class="s">"/apply"</span><span class="o">)</span>
<span class="nd">@RateLimit</span><span class="o">(</span><span class="n">bucket</span> <span class="o">=</span> <span class="s">"apply"</span><span class="o">,</span> <span class="n">limitKey</span> <span class="o">=</span> <span class="s">"APPLY_RATE_LIMIT"</span><span class="o">,</span> <span class="n">defaultLimit</span> <span class="o">=</span> <span class="mi">10</span><span class="o">,</span>
           <span class="n">message</span> <span class="o">=</span> <span class="s">"提交过于频繁，请稍后再试"</span><span class="o">)</span>
<span class="kd">public</span> <span class="nc">ResponseEntity</span><span class="o">&lt;?&gt;</span> <span class="n">apply</span><span class="o">(</span><span class="nd">@RequestBody</span> <span class="nc">Map</span><span class="o">&lt;</span><span class="nc">String</span><span class="o">,</span> <span class="nc">String</span><span class="o">&gt;</span> <span class="n">body</span><span class="o">)</span> <span class="o">{</span> <span class="o">...</span> <span class="o">}</span>
</code></pre></div></div>

<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">public</span> <span class="kt">boolean</span> <span class="nf">preHandle</span><span class="o">(</span><span class="nc">HttpServletRequest</span> <span class="n">req</span><span class="o">,</span> <span class="nc">HttpServletResponse</span> <span class="n">resp</span><span class="o">,</span> <span class="nc">Object</span> <span class="n">handler</span><span class="o">)</span> <span class="o">{</span>
    <span class="nc">RateLimit</span> <span class="n">rl</span> <span class="o">=</span> <span class="n">annotationOf</span><span class="o">(</span><span class="n">handler</span><span class="o">);</span>
    <span class="k">if</span> <span class="o">(</span><span class="n">rl</span> <span class="o">==</span> <span class="kc">null</span><span class="o">)</span> <span class="k">return</span> <span class="kc">true</span><span class="o">;</span>

    <span class="kt">int</span> <span class="n">limit</span> <span class="o">=</span> <span class="n">config</span><span class="o">.</span><span class="na">getInt</span><span class="o">(</span><span class="n">rl</span><span class="o">.</span><span class="na">limitKey</span><span class="o">(),</span> <span class="n">rl</span><span class="o">.</span><span class="na">defaultLimit</span><span class="o">());</span>   <span class="c1">// 每次请求读取，热更新即时生效</span>
    <span class="k">if</span> <span class="o">(</span><span class="n">limit</span> <span class="o">&lt;=</span> <span class="mi">0</span><span class="o">)</span> <span class="k">return</span> <span class="kc">true</span><span class="o">;</span>                                    <span class="c1">// 清空或非正数：停止限流，而不是全部拦截</span>

    <span class="nc">String</span> <span class="n">subject</span> <span class="o">=</span> <span class="n">resolveSubject</span><span class="o">(</span><span class="n">req</span><span class="o">);</span>
    <span class="k">if</span> <span class="o">(</span><span class="n">limiter</span><span class="o">.</span><span class="na">tryAcquire</span><span class="o">(</span><span class="n">rl</span><span class="o">.</span><span class="na">bucket</span><span class="o">()</span> <span class="o">+</span> <span class="s">":"</span> <span class="o">+</span> <span class="n">subject</span><span class="o">,</span> <span class="n">limit</span><span class="o">,</span> <span class="n">windowMillis</span><span class="o">()))</span> <span class="k">return</span> <span class="kc">true</span><span class="o">;</span>

    <span class="n">metrics</span><span class="o">.</span><span class="na">recordRateLimited</span><span class="o">(</span><span class="n">subject</span><span class="o">,</span> <span class="n">rl</span><span class="o">.</span><span class="na">bucket</span><span class="o">());</span>
    <span class="n">writeJson</span><span class="o">(</span><span class="n">resp</span><span class="o">,</span> <span class="mi">429</span><span class="o">,</span> <span class="nc">ApiResponse</span><span class="o">.</span><span class="na">error</span><span class="o">(</span><span class="s">"RATE_LIMITED"</span><span class="o">,</span> <span class="n">rl</span><span class="o">.</span><span class="na">message</span><span class="o">()));</span>
    <span class="k">return</span> <span class="kc">false</span><span class="o">;</span>
<span class="o">}</span>
</code></pre></div></div>

<p><strong>限流的主体优先使用登录身份。</strong> 这样同一个人换浏览器或开多个标签页，消耗的是同一份额度。只有在读不到身份时，才退回到客户端地址，并加上 <code class="language-plaintext highlighter-rouge">ip:</code> 前缀与身份区分。之所以退回到地址而不是归入一个共享桶，是因为共享桶会被任意一个调用方耗尽，进而影响所有无法识别身份的调用方。</p>

<p><strong>限流器是滑动窗口日志。</strong> 每个桶保存一个时间戳队列，在 <code class="language-plaintext highlighter-rouge">ConcurrentHashMap.compute</code> 中完成”清理过期时间戳、判断、追加”，整个过程对单个桶是原子的。桶清空后返回 <code class="language-plaintext highlighter-rouge">null</code>，该键即被移除：</p>

<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">buckets</span><span class="o">.</span><span class="na">compute</span><span class="o">(</span><span class="n">key</span><span class="o">,</span> <span class="o">(</span><span class="n">k</span><span class="o">,</span> <span class="n">q</span><span class="o">)</span> <span class="o">-&gt;</span> <span class="o">{</span>
    <span class="nc">Deque</span><span class="o">&lt;</span><span class="nc">Long</span><span class="o">&gt;</span> <span class="n">d</span> <span class="o">=</span> <span class="o">(</span><span class="n">q</span> <span class="o">==</span> <span class="kc">null</span><span class="o">)</span> <span class="o">?</span> <span class="k">new</span> <span class="nc">ArrayDeque</span><span class="o">&lt;&gt;()</span> <span class="o">:</span> <span class="n">q</span><span class="o">;</span>
    <span class="k">while</span> <span class="o">(!</span><span class="n">d</span><span class="o">.</span><span class="na">isEmpty</span><span class="o">()</span> <span class="o">&amp;&amp;</span> <span class="n">now</span> <span class="o">-</span> <span class="n">d</span><span class="o">.</span><span class="na">peekFirst</span><span class="o">()</span> <span class="o">&gt;=</span> <span class="n">windowMillis</span><span class="o">)</span> <span class="n">d</span><span class="o">.</span><span class="na">pollFirst</span><span class="o">();</span>
    <span class="k">if</span> <span class="o">(</span><span class="n">d</span><span class="o">.</span><span class="na">size</span><span class="o">()</span> <span class="o">&lt;</span> <span class="n">limit</span><span class="o">)</span> <span class="o">{</span> <span class="n">d</span><span class="o">.</span><span class="na">addLast</span><span class="o">(</span><span class="n">now</span><span class="o">);</span> <span class="n">allowed</span><span class="o">[</span><span class="mi">0</span><span class="o">]</span> <span class="o">=</span> <span class="kc">true</span><span class="o">;</span> <span class="o">}</span>
    <span class="k">return</span> <span class="n">d</span><span class="o">.</span><span class="na">isEmpty</span><span class="o">()</span> <span class="o">?</span> <span class="kc">null</span> <span class="o">:</span> <span class="n">d</span><span class="o">;</span>
<span class="o">});</span>
</code></pre></div></div>

<p><strong>取舍需要写进注释。</strong> 限流先于幂等检查执行，所以超出额度的重复提交得到的是 429，而不是”重复提交”。我们接受这一点，并在接口注释中写明，避免后来的维护者误以为这是缺陷。</p>

<h3 id="33-配置的默认值按误配置的后果决定方向">3.3 配置的默认值：按误配置的后果决定方向</h3>

<p>热更新配置随时可能缺失或写错。我们为每个配置项明确规定缺省时的行为，规则是：<strong>看误配置的主要后果是什么，朝后果较轻的方向缺省。</strong></p>

<table>
  <thead>
    <tr>
      <th>配置项</th>
      <th>误配置的主要后果</th>
      <th>缺省行为</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <td>限流阈值</td>
      <td>误伤正常流量</td>
      <td>不限流（阈值 ≤ 0 视为关闭限流）</td>
    </tr>
    <tr>
      <td>受限资源的开放模式</td>
      <td>资源被意外开放</td>
      <td>未知值视为”关闭”</td>
    </tr>
    <tr>
      <td>受限资源的开放时间窗口</td>
      <td>资源被意外开放</td>
      <td>任一字段缺失或格式错误即视为不在窗口内</td>
    </tr>
    <tr>
      <td>鉴权所需的规则码</td>
      <td>越权访问</td>
      <td>缺失即拒绝</td>
    </tr>
    <tr>
      <td>新流程开关</td>
      <td>启用未经验证的路径</td>
      <td>走旧流程</td>
    </tr>
    <tr>
      <td>重试次数、退避时间</td>
      <td>负值导致跳过调用或负数休眠</td>
      <td>下限截断为 0</td>
    </tr>
  </tbody>
</table>

<p>开放模式的解析如下：</p>

<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nc">AccessMode</span> <span class="nf">resolve</span><span class="o">(</span><span class="nc">String</span> <span class="n">raw</span><span class="o">)</span> <span class="o">{</span>
    <span class="k">if</span> <span class="o">(</span><span class="n">raw</span> <span class="o">==</span> <span class="kc">null</span> <span class="o">||</span> <span class="n">raw</span><span class="o">.</span><span class="na">isBlank</span><span class="o">())</span> <span class="k">return</span> <span class="no">TEST_WINDOW</span><span class="o">;</span>        <span class="c1">// 未配置：受限开放（默认）</span>
    <span class="k">return</span> <span class="nf">switch</span> <span class="o">(</span><span class="n">raw</span><span class="o">.</span><span class="na">trim</span><span class="o">().</span><span class="na">toUpperCase</span><span class="o">(</span><span class="nc">Locale</span><span class="o">.</span><span class="na">ROOT</span><span class="o">))</span> <span class="o">{</span>
        <span class="k">case</span> <span class="s">"OPEN"</span>        <span class="o">-&gt;</span> <span class="no">OPEN</span><span class="o">;</span>
        <span class="k">case</span> <span class="s">"TEST_WINDOW"</span> <span class="o">-&gt;</span> <span class="no">TEST_WINDOW</span><span class="o">;</span>
        <span class="k">default</span>            <span class="o">-&gt;</span> <span class="no">CLOSED</span><span class="o">;</span>                            <span class="c1">// 拼写错误等任何未知值：关闭</span>
    <span class="o">};</span>
<span class="o">}</span>
</code></pre></div></div>

<p>开放时间窗口按固定时区解析，而不是使用 JVM 默认时区，保证同一份配置在每台主机上对应同一段时间。</p>

<h3 id="34-鉴权失败即拒绝拒绝时不泄露信息">3.4 鉴权：失败即拒绝，拒绝时不泄露信息</h3>

<ul>
  <li><strong>权限服务调用的任何异常都按”无权限”处理</strong>，规则码缺失同样按”无权限”处理。</li>
  <li><strong>一次往返校验多个规则码</strong>，用于”管理员也可以访问”这类组合判断。</li>
  <li><strong>区分 401 与 403。</strong> 读不到登录身份，或登录 SDK 抛出异常，返回 401；身份有效但无权限，或权限服务异常，返回 403。</li>
  <li><strong>403 的响应消息固定为 “Access denied”</strong>，被拒绝的调用方无法得知需要哪一个权限。</li>
  <li><strong>运维控制台的操作者身份由服务端写入。</strong> 服务端先复制客户端提交的参数，再用登录身份覆盖保留的 <code class="language-plaintext highlighter-rouge">__authenticatedOperator</code> 参数。顺序不能颠倒，否则客户端可以通过提交同名参数伪造操作者。审计日志只记录经过脱敏和长度截断的参数。</li>
</ul>

<h3 id="35-错误响应对客户端提供可操作的信息细节留在日志">3.5 错误响应：对客户端提供可操作的信息，细节留在日志</h3>

<p>我们在接口层区分两类错误：</p>

<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1">// 客户端错误：返回一条可操作的提示，具体细节只写日志</span>
<span class="nc">ResponseEntity</span><span class="o">&lt;?&gt;</span> <span class="n">clientFacingError</span><span class="o">(</span><span class="nc">String</span> <span class="n">code</span><span class="o">,</span> <span class="nc">String</span> <span class="n">userMessage</span><span class="o">,</span> <span class="nc">String</span> <span class="n">logDetail</span><span class="o">)</span> <span class="o">{</span>
    <span class="n">log</span><span class="o">.</span><span class="na">warn</span><span class="o">(</span><span class="s">"rejected code={} detail={}"</span><span class="o">,</span> <span class="n">code</span><span class="o">,</span> <span class="n">logDetail</span><span class="o">);</span>
    <span class="k">return</span> <span class="nc">ApiResponse</span><span class="o">.</span><span class="na">error</span><span class="o">(</span><span class="n">code</span><span class="o">,</span> <span class="n">userMessage</span><span class="o">);</span>
<span class="o">}</span>

<span class="c1">// 服务端错误：日志记录原因和堆栈，客户端只收到通用提示</span>
<span class="nc">ResponseEntity</span><span class="o">&lt;?&gt;</span> <span class="n">internalError</span><span class="o">(</span><span class="nc">String</span> <span class="n">code</span><span class="o">,</span> <span class="nc">String</span> <span class="n">logDetail</span><span class="o">,</span> <span class="nc">Throwable</span> <span class="n">cause</span><span class="o">)</span> <span class="o">{</span>
    <span class="n">log</span><span class="o">.</span><span class="na">error</span><span class="o">(</span><span class="s">"internal code={} detail={}"</span><span class="o">,</span> <span class="n">code</span><span class="o">,</span> <span class="n">logDetail</span><span class="o">,</span> <span class="n">cause</span><span class="o">);</span>
    <span class="k">return</span> <span class="nc">ApiResponse</span><span class="o">.</span><span class="na">error</span><span class="o">(</span><span class="n">code</span><span class="o">,</span> <span class="no">GENERIC_ERROR_MESSAGE</span><span class="o">);</span>
<span class="o">}</span>
</code></pre></div></div>

<p>状态码按”客户端接下来应该做什么”来设计，而不是按”服务端内部发生了什么”：</p>

<ul>
  <li><strong><code class="language-plaintext highlighter-rouge">DUPLICATE_SUBMIT</code> 与 <code class="language-plaintext highlighter-rouge">ACCEPTED</code> 分开。</strong> 两者的 HTTP 状态都是 202，但前端收到 <code class="language-plaintext highlighter-rouge">DUPLICATE_SUBMIT</code> 时会保持表单锁定，而不是把它当作一次新的受理。</li>
  <li><strong>“等待人工处理”按能否自行恢复拆成两种提示。</strong> 如果请求停在一个可以对原 MR 重试的步骤上，提示”可以继续等待”；如果停在确定性的外部失败上（例如流水线失败），提示”需要联系维护团队，继续等待不会生效”。哪些步骤属于后者，由一个常量集合集中定义。</li>
  <li><strong>只读的预览接口失败时返回空数据</strong>，不阻塞用户继续提交。</li>
</ul>

<h3 id="36-幂等与长操作">3.6 幂等与长操作</h3>

<p><strong>幂等。</strong> 写接口以客户端提供的 <code class="language-plaintext highlighter-rouge">operationId</code> 作为幂等键，依赖数据库唯一约束完成”查询、插入、冲突后重读”（详见数据库篇 3.4 节）。并发的重复请求中，只有一个会继续执行，其余的返回胜出方的结果。</p>

<p><strong>长操作。</strong> 长操作在完成同步校验后返回 HTTP 202，后续在有界线程池中执行。线程池拒绝任务时，失败状态会写入数据库（详见 MR 生命周期篇 3.2 节）。客户端始终能通过轮询看到一个明确的状态。</p>

<h3 id="37-可观测性不干扰业务">3.7 可观测性不干扰业务</h3>

<p>我们为指标代码制定了几条规则，并在实现中逐条落实：</p>

<ul>
  <li><strong>指标拦截器的 <code class="language-plaintext highlighter-rouge">preHandle</code> 永远返回 <code class="language-plaintext highlighter-rouge">true</code>，从不写响应。</strong></li>
  <li><strong>指标代码只读取已经解析好的值。</strong> 例如读取请求参数时使用 <code class="language-plaintext highlighter-rouge">request.getParameter</code>，而不是自己读请求体，避免消耗输入流、影响后续的参数绑定。</li>
  <li><strong>指标使用独立的身份解析逻辑。</strong> 身份不可信时（例如只能拿到一个可能是工号的裸 ID），直接跳过本次计数，不做猜测，解析结果也不回流到业务逻辑。</li>
  <li><strong>计数器写入包裹在 <code class="language-plaintext highlighter-rouge">try/catch</code> 中</strong>，异常只记日志。标签值经过规范化，并截断到 128 个字符；维度为空或非法时跳过发送。</li>
  <li><strong>按语义选择计数时机。</strong> “收到请求”在请求到达时计数；”成功访问”在 <code class="language-plaintext highlighter-rouge">afterCompletion</code> 中计数，并且只统计 2xx 响应。</li>
  <li><strong>业务结果指标在事务提交后才发送</strong>，并且只在状态真正发生翻转时发送，避免回滚或并发导致多计（详见 MR 生命周期篇 3.7 节）。</li>
  <li><strong>指标代码放在失败处理逻辑之外。</strong> 例如批次移交完成后才记录耗时，这样指标代码即使抛出异常，也不会触发批次的失败结算。</li>
</ul>

<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">public</span> <span class="kt">void</span> <span class="nf">incrementUserCounter</span><span class="o">(</span><span class="nc">String</span> <span class="n">name</span><span class="o">,</span> <span class="nc">Map</span><span class="o">&lt;</span><span class="nc">String</span><span class="o">,</span> <span class="nc">String</span><span class="o">&gt;</span> <span class="n">tags</span><span class="o">)</span> <span class="o">{</span>
    <span class="k">try</span> <span class="o">{</span>
        <span class="nc">Map</span><span class="o">&lt;</span><span class="nc">String</span><span class="o">,</span> <span class="nc">String</span><span class="o">&gt;</span> <span class="n">safe</span> <span class="o">=</span> <span class="n">normalizeAndTruncate</span><span class="o">(</span><span class="n">tags</span><span class="o">,</span> <span class="mi">128</span><span class="o">);</span>
        <span class="k">if</span> <span class="o">(</span><span class="n">safe</span> <span class="o">==</span> <span class="kc">null</span><span class="o">)</span> <span class="o">{</span> <span class="n">log</span><span class="o">.</span><span class="na">warn</span><span class="o">(</span><span class="s">"skip metric {}: invalid tags"</span><span class="o">,</span> <span class="n">name</span><span class="o">);</span> <span class="k">return</span><span class="o">;</span> <span class="o">}</span>
        <span class="n">registry</span><span class="o">.</span><span class="na">counter</span><span class="o">(</span><span class="n">name</span><span class="o">,</span> <span class="n">safe</span><span class="o">).</span><span class="na">inc</span><span class="o">();</span>
    <span class="o">}</span> <span class="k">catch</span> <span class="o">(</span><span class="nc">RuntimeException</span> <span class="n">e</span><span class="o">)</span> <span class="o">{</span>
        <span class="n">log</span><span class="o">.</span><span class="na">error</span><span class="o">(</span><span class="s">"metric {} failed; business request continues"</span><span class="o">,</span> <span class="n">name</span><span class="o">,</span> <span class="n">e</span><span class="o">);</span>   <span class="c1">// 指标失败不影响业务</span>
    <span class="o">}</span>
<span class="o">}</span>
</code></pre></div></div>

<h3 id="38-前端发布期间的静态资源回退">3.8 前端发布期间的静态资源回退</h3>

<p>SPA 发布之后，已打开的旧页面会请求新版本中已被删除的带 hash 资源。如果不做处理，这类请求会落到全局异常处理器，返回 500。我们从服务端和客户端两侧处理：</p>

<p><strong>服务端：</strong></p>

<ul>
  <li><code class="language-plaintext highlighter-rouge">index.html</code> 设置 <code class="language-plaintext highlighter-rouge">Cache-Control: no-store</code>，保证每次都拿到最新的资源清单；</li>
  <li>带 hash 的资源设置 <code class="language-plaintext highlighter-rouge">max-age=365d, public, immutable</code>；</li>
  <li>资源不存在时按请求类型分别响应：页面导航请求（<code class="language-plaintext highlighter-rouge">Accept: text/html</code> 且路径不含扩展名）返回一个回退页面，静态资源或 API 请求返回 404 JSON。两种响应都设置 <code class="language-plaintext highlighter-rouge">no-store</code>。</li>
</ul>

<p><strong>客户端：</strong></p>

<ul>
  <li>在捕获阶段监听脚本和样式的加载错误；核心资源加载失败时，带上缓存破坏参数重新加载页面；</li>
  <li>用 <code class="language-plaintext highlighter-rouge">sessionStorage</code> 记录上次自动重载的时间。30 秒内不再自动重载，改为显示一个”刷新页面”按钮，避免陷入重载循环。</li>
</ul>

<p>我们还为这些响应头和状态码编写了冒烟脚本，在预发布环境验证。</p>

<h2 id="4-成果">4. 成果</h2>

<ul>
  <li><strong>每个横切关注点的缺省行为都有明确定义</strong>，并写在代码注释和配置说明中。配置被清空或写错时，限流退化为不限流，开放类开关退化为关闭，鉴权退化为拒绝。</li>
  <li><strong>限流和鉴权在参数绑定之前完成</strong>，不需要修改任何既有接口的返回结构。</li>
  <li><strong>被限流的请求不会污染业务指标</strong>，它们由独立的指标统计。</li>
  <li><strong>指标代码中的异常不影响业务请求。</strong> 指标不读取请求体，不写响应，也不参与业务分支判断。</li>
  <li><strong>错误响应按”下一步行动”区分。</strong> 用户能区分”重复提交”“可以继续等待”和”需要联系团队”，内部细节只出现在服务端日志里。</li>
  <li><strong>前端发布期间，旧页面能自行恢复</strong>，不再出现由缺失资源引起的 500。</li>
</ul>

<h2 id="5-已知限制与后续计划">5. 已知限制与后续计划</h2>

<ul>
  <li><strong>限流按实例生效。</strong> 实际的全局额度约等于”单实例阈值 × 实例数”。另外，退回到地址限流时取的是 <code class="language-plaintext highlighter-rouge">X-Forwarded-For</code> 的第一跳，而这个值客户端可以伪造。后续计划改为取可信代理追加的那一跳，或在网关层统一限流。</li>
  <li><strong>身份解析逻辑有三份副本，规则存在细微差异。</strong> 其中一份对邮箱形式的账号不做截断，可能导致同一个人在指标中出现两个标签值。后续计划收敛为一个共享的解析组件。</li>
  <li><strong>异常与 HTTP 状态码的映射不一致。</strong> 部分业务异常返回 500；有一处鉴权拒绝经由通用服务异常返回 500 而不是 403；兜底处理器把异常消息原样返回给客户端。后续计划统一错误模型，兜底处理器只返回通用提示。</li>
  <li><strong>部分外部 SDK 调用没有设置超时，也没有熔断。</strong> 外部依赖变慢时会长时间占用请求线程。后续计划为所有外部调用设置显式超时，并为关键依赖增加隔离。</li>
  <li><strong>幂等只记录 <code class="language-plaintext highlighter-rouge">operationId</code>，不重放原始响应。</strong> 如果客户端没有提供 <code class="language-plaintext highlighter-rouge">operationId</code>，服务端生成的 ID 含有时间戳，重试请求无法去重。</li>
  <li><strong>API 指标拦截器自身没有 <code class="language-plaintext highlighter-rouge">try/catch</code>。</strong> 它能否做到失败不影响业务，取决于追踪和指标库本身不抛异常。另外，部分用户接口不在 API 指标的覆盖范围内。</li>
</ul>]]></content><author><name></name></author><category term="zh" /><summary type="html"><![CDATA[为限流、功能开关、鉴权与指标等横切关注点定义依赖缺失、配置错误和依赖失败时的缺省行为，让误配置朝后果较轻的方向退化。]]></summary></entry><entry xml:lang="zh-CN"><title type="html">让 MySQL 守住业务不变量：生成列单活槽位、守卫式 UPDATE 与”重复键即答案”</title><link href="https://blog.changcodes.com/zh/posts/mysql-business-invariants/" rel="alternate" type="text/html" title="让 MySQL 守住业务不变量：生成列单活槽位、守卫式 UPDATE 与”重复键即答案”" /><published>2026-05-17T00:00:00+00:00</published><updated>2026-05-17T00:00:00+00:00</updated><id>https://blog.changcodes.com/zh/posts/mysql-business-invariants</id><content type="html" xml:base="https://blog.changcodes.com/zh/posts/mysql-business-invariants/"><![CDATA[<h2 id="1-问题与目标">1. 问题与目标</h2>

<p>我们的系统通过向 Git 仓库发起 Merge Request（MR）来落地用户的变更申请。如果每个申请各自发起一个 MR，多个 MR 会同时修改同一个仓库，彼此导致 rebase，并在流水线上排队。为此我们引入了<strong>批量聚合</strong>：申请审批通过后先进入等待集合，每一轮把等待中的申请合并成一个 MR 统一处理。</p>

<p>这个设计要求以下不变量始终成立：</p>

<ul>
  <li><strong>I1 单活批次</strong>：任意时刻最多只有一个活跃批次（运行中或阻塞中）。</li>
  <li><strong>I2 不重复认领</strong>：一个申请最多被一个批次认领。</li>
  <li><strong>I3 入队不阻塞</strong>：批次运行期间，新申请可以继续进入等待集合。</li>
  <li><strong>I4 入队幂等</strong>：同一个申请重复提交，只入队一次。</li>
  <li><strong>I5 一关系一行</strong>：每个”主体—目标”关系只有一行当前状态，重新申请复用这一行。</li>
  <li><strong>I6 过期写入失败</strong>：状态转移基于预期的当前状态，状态已被别处改动时，写入必须失败。</li>
</ul>

<p>目标是让数据库直接保证这些不变量，而不是依赖”应用代码恰好没有写错”。</p>

<h2 id="2-约束与考量">2. 约束与考量</h2>

<ul>
  <li><strong>数据库是 MySQL InnoDB。</strong> MySQL 不支持 PostgreSQL 那样的部分唯一索引（<code class="language-plaintext highlighter-rouge">CREATE UNIQUE INDEX ... WHERE status = 'RUNNING'</code>）。</li>
  <li><strong>服务多实例部署，不引入新组件。</strong> 没有 Redis 或 ZooKeeper 之类的分布式锁可用，也不打算为此引入。</li>
  <li><strong>触发源会重叠。</strong> 调度平台保证定时任务单实例执行，但运维人员可以手动触发同一轮次，手动触发与定时触发可能同时发生。</li>
  <li><strong>批次持续时间长。</strong> 一个批次从创建 MR 到合并需要数分钟，不能在这段时间里持有数据库锁。</li>
  <li><strong>进程可能在任意一步崩溃。</strong> 互斥机制不能因为持有者崩溃而永久泄漏。</li>
</ul>

<h2 id="3-方案">3. 方案</h2>

<h3 id="31-生成列加唯一索引单活槽位i1">3.1 生成列加唯一索引：单活槽位（I1）</h3>

<div class="language-sql highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">CREATE</span> <span class="k">TABLE</span> <span class="n">merge_batch</span> <span class="p">(</span>
  <span class="n">id</span>          <span class="nb">BIGINT</span>      <span class="k">NOT</span> <span class="k">NULL</span> <span class="n">AUTO_INCREMENT</span><span class="p">,</span>
  <span class="n">status</span>      <span class="nb">TINYINT</span>     <span class="k">NOT</span> <span class="k">NULL</span> <span class="k">DEFAULT</span> <span class="mi">1</span> <span class="k">COMMENT</span> <span class="s1">'1=RUNNING 2=SUCCEEDED 3=FAILED 4=BLOCKED'</span><span class="p">,</span>
  <span class="n">active_slot</span> <span class="nb">TINYINT</span> <span class="k">GENERATED</span> <span class="n">ALWAYS</span> <span class="k">AS</span>
                <span class="p">(</span><span class="k">CASE</span> <span class="k">WHEN</span> <span class="n">status</span> <span class="k">IN</span> <span class="p">(</span><span class="mi">1</span><span class="p">,</span> <span class="mi">4</span><span class="p">)</span> <span class="k">THEN</span> <span class="mi">1</span> <span class="k">ELSE</span> <span class="k">NULL</span> <span class="k">END</span><span class="p">)</span> <span class="n">STORED</span><span class="p">,</span>
  <span class="n">mr_id</span>       <span class="nb">VARCHAR</span><span class="p">(</span><span class="mi">64</span><span class="p">)</span> <span class="k">NULL</span><span class="p">,</span>
  <span class="n">created_at</span>  <span class="nb">DATETIME</span>    <span class="k">NOT</span> <span class="k">NULL</span> <span class="k">DEFAULT</span> <span class="k">CURRENT_TIMESTAMP</span><span class="p">,</span>
  <span class="k">PRIMARY</span> <span class="k">KEY</span> <span class="p">(</span><span class="n">id</span><span class="p">),</span>
  <span class="k">UNIQUE</span> <span class="k">KEY</span> <span class="n">uk_active_slot</span> <span class="p">(</span><span class="n">active_slot</span><span class="p">),</span>
  <span class="k">KEY</span> <span class="n">idx_status_created</span> <span class="p">(</span><span class="n">status</span><span class="p">,</span> <span class="n">created_at</span><span class="p">,</span> <span class="n">id</span><span class="p">)</span>
<span class="p">)</span> <span class="n">ENGINE</span> <span class="o">=</span> <span class="n">InnoDB</span><span class="p">;</span>
</code></pre></div></div>

<p>这个设计利用了 MySQL 唯一索引的一条语义：<strong>唯一索引允许多个 NULL</strong>。</p>

<ul>
  <li>活跃批次（RUNNING、BLOCKED）的 <code class="language-plaintext highlighter-rouge">active_slot</code> 为 1，唯一索引保证全表最多一行取值为 1。</li>
  <li>终态批次（SUCCEEDED、FAILED）的 <code class="language-plaintext highlighter-rouge">active_slot</code> 为 NULL，任意多行可以共存。</li>
  <li>批次进入终态的那条 <code class="language-plaintext highlighter-rouge">UPDATE status</code> 同时让 <code class="language-plaintext highlighter-rouge">active_slot</code> 变成 NULL，槽位随之释放，不需要额外的释放步骤。</li>
</ul>

<p>这在效果上等价于部分唯一索引 <code class="language-plaintext highlighter-rouge">UNIQUE (status) WHERE status IN (RUNNING, BLOCKED)</code>。</p>

<p>我们也比较了其他互斥方案：</p>

<table>
  <thead>
    <tr>
      <th>方案</th>
      <th>主要问题</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <td>锁行加 <code class="language-plaintext highlighter-rouge">SELECT ... FOR UPDATE</code></td>
      <td>锁要么在整个批次期间持有（数分钟的长事务），要么另外维护一个租约字段</td>
    </tr>
    <tr>
      <td><code class="language-plaintext highlighter-rouge">GET_LOCK()</code> 会话级锁</td>
      <td>锁绑定在数据库连接上，与连接池的连接复用不易配合</td>
    </tr>
    <tr>
      <td>Redis 等外部分布式锁</td>
      <td>需要新组件；租约过期时间与任务实际耗时难以对齐</td>
    </tr>
    <tr>
      <td>生成列加唯一索引</td>
      <td>锁与状态是同一列，不存在租约，也不会出现”状态已终结而锁未释放”</td>
    </tr>
  </tbody>
</table>

<p>最后一行是我们选择它的原因：互斥状态就是业务状态本身。持有者崩溃时，批次停在 RUNNING，槽位保持占用，但这是一个可以被查询、被处理的显式状态，而不是一个看不见的泄漏锁。超时扫描或运维工具把它推进到终态，槽位即释放。阻塞态（BLOCKED）同样占用槽位，后续批次会停下来，直到这个批次被处理。</p>

<h3 id="32-认领事务不加锁的读取与守卫式-updatei2i3">3.2 认领事务：不加锁的读取与守卫式 UPDATE（I2、I3）</h3>

<div class="language-sql highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">CREATE</span> <span class="k">TABLE</span> <span class="n">merge_batch_request</span> <span class="p">(</span>
  <span class="n">id</span>           <span class="nb">BIGINT</span>      <span class="k">NOT</span> <span class="k">NULL</span> <span class="n">AUTO_INCREMENT</span><span class="p">,</span>
  <span class="n">operation_id</span> <span class="nb">VARCHAR</span><span class="p">(</span><span class="mi">36</span><span class="p">)</span> <span class="k">NOT</span> <span class="k">NULL</span><span class="p">,</span>
  <span class="n">batch_id</span>     <span class="nb">BIGINT</span>      <span class="k">NULL</span> <span class="k">COMMENT</span> <span class="s1">'NULL 表示仍在等待'</span><span class="p">,</span>
  <span class="n">created_at</span>   <span class="nb">DATETIME</span>    <span class="k">NOT</span> <span class="k">NULL</span> <span class="k">DEFAULT</span> <span class="k">CURRENT_TIMESTAMP</span><span class="p">,</span>
  <span class="k">PRIMARY</span> <span class="k">KEY</span> <span class="p">(</span><span class="n">id</span><span class="p">),</span>
  <span class="k">UNIQUE</span> <span class="k">KEY</span> <span class="n">uk_operation_id</span> <span class="p">(</span><span class="n">operation_id</span><span class="p">),</span>
  <span class="k">KEY</span> <span class="n">idx_batch_created</span> <span class="p">(</span><span class="n">batch_id</span><span class="p">,</span> <span class="n">created_at</span><span class="p">,</span> <span class="n">id</span><span class="p">)</span>
<span class="p">)</span> <span class="n">ENGINE</span> <span class="o">=</span> <span class="n">InnoDB</span><span class="p">;</span>
</code></pre></div></div>

<p>每一轮的认领在一个事务中完成：</p>

<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nd">@Transactional</span>
<span class="kd">public</span> <span class="nc">Claim</span> <span class="nf">claim</span><span class="o">(</span><span class="kt">int</span> <span class="n">maxRequests</span><span class="o">)</span> <span class="o">{</span>
    <span class="c1">// 1. 占用槽位</span>
    <span class="kt">long</span> <span class="n">batchId</span><span class="o">;</span>
    <span class="k">try</span> <span class="o">{</span>
        <span class="n">batchId</span> <span class="o">=</span> <span class="n">insertReturningKey</span><span class="o">(</span><span class="s">"INSERT INTO merge_batch (status) VALUES (1)"</span><span class="o">);</span>
    <span class="o">}</span> <span class="k">catch</span> <span class="o">(</span><span class="nc">DuplicateKeyException</span> <span class="n">e</span><span class="o">)</span> <span class="o">{</span>
        <span class="k">return</span> <span class="nc">Claim</span><span class="o">.</span><span class="na">slotOccupied</span><span class="o">();</span>   <span class="c1">// 本事务尚未写入任何数据，正常返回等价于回滚</span>
    <span class="o">}</span>

    <span class="c1">// 2. 读取等待中的申请：普通读，不加 FOR UPDATE</span>
    <span class="nc">List</span><span class="o">&lt;</span><span class="nc">Req</span><span class="o">&gt;</span> <span class="n">pending</span> <span class="o">=</span> <span class="n">query</span><span class="o">(</span>
        <span class="s">"SELECT id, operation_id FROM merge_batch_request "</span> <span class="o">+</span>
        <span class="s">"WHERE batch_id IS NULL AND created_at &lt; CURRENT_TIMESTAMP "</span> <span class="o">+</span>
        <span class="s">"ORDER BY created_at, id LIMIT ?"</span><span class="o">,</span> <span class="n">maxRequests</span><span class="o">);</span>
    <span class="k">if</span> <span class="o">(</span><span class="n">pending</span><span class="o">.</span><span class="na">isEmpty</span><span class="o">())</span> <span class="o">{</span>
        <span class="k">throw</span> <span class="k">new</span> <span class="nf">NothingToClaimException</span><span class="o">();</span>   <span class="c1">// 回滚，释放刚占用的槽位</span>
    <span class="o">}</span>

    <span class="c1">// 3. 守卫式认领：只认领仍未被认领的行，且影响行数必须与读到的行数一致</span>
    <span class="kt">int</span> <span class="n">updated</span> <span class="o">=</span> <span class="n">update</span><span class="o">(</span>
        <span class="s">"UPDATE merge_batch_request SET batch_id = ? "</span> <span class="o">+</span>
        <span class="s">"WHERE batch_id IS NULL AND id IN ("</span> <span class="o">+</span> <span class="n">placeholders</span><span class="o">(</span><span class="n">pending</span><span class="o">)</span> <span class="o">+</span> <span class="s">")"</span><span class="o">,</span>
        <span class="n">batchId</span><span class="o">,</span> <span class="n">ids</span><span class="o">(</span><span class="n">pending</span><span class="o">));</span>
    <span class="k">if</span> <span class="o">(</span><span class="n">updated</span> <span class="o">!=</span> <span class="n">pending</span><span class="o">.</span><span class="na">size</span><span class="o">())</span> <span class="o">{</span>
        <span class="k">throw</span> <span class="k">new</span> <span class="nf">ClaimPreemptedException</span><span class="o">();</span>   <span class="c1">// 有行被别处改动，整体回滚</span>
    <span class="o">}</span>

    <span class="c1">// 4. 写入只追加的成员历史</span>
    <span class="n">insertHistory</span><span class="o">(</span><span class="n">batchId</span><span class="o">,</span> <span class="n">pending</span><span class="o">);</span>
    <span class="k">return</span> <span class="nc">Claim</span><span class="o">.</span><span class="na">claimed</span><span class="o">(</span><span class="n">batchId</span><span class="o">,</span> <span class="n">pending</span><span class="o">);</span>
<span class="o">}</span>
</code></pre></div></div>

<p><strong>为什么第 2 步不用 <code class="language-plaintext highlighter-rouge">FOR UPDATE</code>。</strong> 在 REPEATABLE READ 隔离级别下，带范围条件的锁定读会对扫描到的索引区间加 next-key lock。这里扫描的是 <code class="language-plaintext highlighter-rouge">idx_batch_created</code> 上 <code class="language-plaintext highlighter-rouge">batch_id IS NULL</code> 的区间，而新入队的申请恰好插入这个区间，于是会被阻塞到认领事务结束，违反 I3。去掉锁定读并不影响正确性：</p>

<ul>
  <li>其他认领者被第 1 步的槽位挡在外面；</li>
  <li>即使出现意外的并发修改，第 3 步的 <code class="language-plaintext highlighter-rouge">WHERE batch_id IS NULL</code> 和影响行数校验也会发现，并让整个事务回滚。</li>
</ul>

<p><strong>为什么读到空集要抛异常。</strong> 如果正常提交，就会留下一个没有任何成员、却占着槽位的 RUNNING 批次。抛异常让槽位随事务一起回滚。</p>

<p><strong>事务外的无锁预检。</strong> 进入认领事务之前，先执行一次 <code class="language-plaintext highlighter-rouge">SELECT ... WHERE batch_id IS NULL ... LIMIT 1</code>。等待集合为空时直接结束本轮，不会出现”占用槽位后又回滚”的空转。</p>

<p><strong>统一使用数据库时钟。</strong> 判断条件写成 <code class="language-plaintext highlighter-rouge">created_at &lt; CURRENT_TIMESTAMP</code>，超时扫描写成 <code class="language-plaintext highlighter-rouge">created_at &lt; DATE_SUB(CURRENT_TIMESTAMP, INTERVAL ? SECOND)</code>。写入时间和比较时间来自同一个时钟，应用服务器之间的时钟偏差不会影响判断。</p>

<p><strong>实现时需要确认的一点。</strong> 第 1 步在事务内捕获了唯一键冲突后正常返回。在 MySQL 中，单条语句失败只回滚这条语句，事务本身仍然可用。但部分数据访问框架会在异常被翻译时把事务标记为 rollback-only，导致提交时报错。采用这一写法前，需要确认所用框架的行为。</p>

<h3 id="33-状态即版本守卫式状态转移i6">3.3 状态即版本：守卫式状态转移（I6）</h3>

<p>所有状态写入都带上预期的当前状态，并检查影响行数：</p>

<div class="language-sql highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1">-- 批次终态转移：只有仍处于 RUNNING 的批次才能转移</span>
<span class="k">UPDATE</span> <span class="n">merge_batch</span> <span class="k">SET</span> <span class="n">status</span> <span class="o">=</span> <span class="mi">2</span> <span class="k">WHERE</span> <span class="n">id</span> <span class="o">=</span> <span class="o">?</span> <span class="k">AND</span> <span class="n">status</span> <span class="o">=</span> <span class="mi">1</span><span class="p">;</span>   <span class="c1">-- 影响行数必须为 1</span>

<span class="c1">-- 普通行的乐观并发控制：以刚读到的状态作为版本</span>
<span class="k">UPDATE</span> <span class="k">operation</span> <span class="k">SET</span> <span class="n">status</span> <span class="o">=</span> <span class="o">?</span><span class="p">,</span> <span class="p">...</span> <span class="k">WHERE</span> <span class="n">operation_id</span> <span class="o">=</span> <span class="o">?</span> <span class="k">AND</span> <span class="n">status</span> <span class="o">=</span> <span class="o">?</span><span class="p">;</span>
</code></pre></div></div>

<p>影响行数为 0 时，表示状态已被别处改动，写入方抛出”被抢占”异常。</p>

<p>这类守卫写入在组合事务中的<strong>位置</strong>也很重要。批次失败时的结算事务包含多步：把批次标记为 FAILED，再删除或重置该批次下每个申请的记录。我们把批次的守卫写入放在第一步：</p>

<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">tx</span><span class="o">.</span><span class="na">execute</span><span class="o">(</span><span class="nc">List</span><span class="o">.</span><span class="na">of</span><span class="o">(</span>
    <span class="o">()</span> <span class="o">-&gt;</span> <span class="n">batchRepo</span><span class="o">.</span><span class="na">markFailed</span><span class="o">(</span><span class="n">batchId</span><span class="o">,</span> <span class="n">code</span><span class="o">,</span> <span class="n">msg</span><span class="o">),</span>  <span class="c1">// WHERE status = RUNNING，影响行数须为 1</span>
    <span class="o">()</span> <span class="o">-&gt;</span> <span class="n">resetOperations</span><span class="o">(</span><span class="n">batchId</span><span class="o">)</span>                   <span class="c1">// 后续步骤</span>
<span class="o">));</span>
</code></pre></div></div>

<p>如果这个批次已经被另一条路径结算过（例如超时扫描与运行中的任务同时到达），第一步就会失败，整个事务回滚，后续的重置不会执行第二遍。</p>

<h3 id="34-把重复键当作答案而不是错误i4">3.4 把重复键当作答案，而不是错误（I4）</h3>

<p>唯一约束冲突在不同场景下有明确的业务含义，我们分别处理：</p>

<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1">// 入队：重复表示"已经入队过"，返回 false，不报错</span>
<span class="kd">public</span> <span class="kt">boolean</span> <span class="nf">admit</span><span class="o">(</span><span class="nc">String</span> <span class="n">opId</span><span class="o">)</span> <span class="o">{</span>
    <span class="k">try</span> <span class="o">{</span>
        <span class="n">insert</span><span class="o">(</span><span class="s">"INSERT INTO merge_batch_request (operation_id) VALUES (?)"</span><span class="o">,</span> <span class="n">opId</span><span class="o">);</span>
        <span class="k">return</span> <span class="kc">true</span><span class="o">;</span>
    <span class="o">}</span> <span class="k">catch</span> <span class="o">(</span><span class="nc">DuplicateKeyException</span> <span class="n">e</span><span class="o">)</span> <span class="o">{</span>
        <span class="k">return</span> <span class="kc">false</span><span class="o">;</span>
    <span class="o">}</span>
<span class="o">}</span>

<span class="c1">// 审计记录：重复表示"已经记录过"，不覆盖已有行（它可能已经是终态）</span>
<span class="kd">public</span> <span class="kt">void</span> <span class="nf">insertAudit</span><span class="o">(</span><span class="nc">Audit</span> <span class="n">a</span><span class="o">)</span> <span class="o">{</span>
    <span class="k">try</span> <span class="o">{</span> <span class="n">insert</span><span class="o">(</span><span class="n">a</span><span class="o">);</span> <span class="o">}</span> <span class="k">catch</span> <span class="o">(</span><span class="nc">DuplicateKeyException</span> <span class="n">e</span><span class="o">)</span> <span class="o">{</span> <span class="cm">/* 预期内的去重结果 */</span> <span class="o">}</span>
<span class="o">}</span>

<span class="c1">// 请求幂等：先查；查不到则插入；插入冲突说明并发请求先插入了，重读并返回胜出方的结果</span>
<span class="kd">public</span> <span class="nc">Result</span> <span class="nf">checkOrInsert</span><span class="o">(</span><span class="nc">String</span> <span class="n">requestId</span><span class="o">,</span> <span class="nc">String</span> <span class="n">opId</span><span class="o">)</span> <span class="o">{</span>
    <span class="nc">Record</span> <span class="n">r</span> <span class="o">=</span> <span class="n">findById</span><span class="o">(</span><span class="n">requestId</span><span class="o">);</span>
    <span class="k">if</span> <span class="o">(</span><span class="n">r</span> <span class="o">!=</span> <span class="kc">null</span><span class="o">)</span> <span class="k">return</span> <span class="nc">Result</span><span class="o">.</span><span class="na">duplicate</span><span class="o">(</span><span class="n">r</span><span class="o">.</span><span class="na">operationId</span><span class="o">());</span>
    <span class="k">try</span> <span class="o">{</span>
        <span class="n">insert</span><span class="o">(</span><span class="k">new</span> <span class="nc">Record</span><span class="o">(</span><span class="n">requestId</span><span class="o">,</span> <span class="n">opId</span><span class="o">));</span>
        <span class="k">return</span> <span class="nc">Result</span><span class="o">.</span><span class="na">fresh</span><span class="o">();</span>
    <span class="o">}</span> <span class="k">catch</span> <span class="o">(</span><span class="nc">DuplicateKeyException</span> <span class="n">e</span><span class="o">)</span> <span class="o">{</span>
        <span class="nc">Record</span> <span class="n">winner</span> <span class="o">=</span> <span class="n">findById</span><span class="o">(</span><span class="n">requestId</span><span class="o">);</span>
        <span class="k">return</span> <span class="n">winner</span> <span class="o">!=</span> <span class="kc">null</span> <span class="o">?</span> <span class="nc">Result</span><span class="o">.</span><span class="na">duplicate</span><span class="o">(</span><span class="n">winner</span><span class="o">.</span><span class="na">operationId</span><span class="o">())</span>
                              <span class="o">:</span> <span class="nc">Result</span><span class="o">.</span><span class="na">retryLater</span><span class="o">();</span>
    <span class="o">}</span>
<span class="o">}</span>
</code></pre></div></div>

<p>这三种写法都没有显式加锁，并发下的正确性来自唯一约束本身。</p>

<h3 id="35-一关系一行原地重置与拒绝覆盖i5">3.5 一关系一行：原地重置与拒绝覆盖（I5）</h3>

<p><code class="language-plaintext highlighter-rouge">binding</code> 表以 <code class="language-plaintext highlighter-rouge">(subject_id, target_key)</code> 为唯一键，每个关系只有一行。但每次申请会生成一个新的 <code class="language-plaintext highlighter-rouge">operation_id</code>，于是”每次操作一行”的写入模型与”每个关系一行”的唯一键之间需要协调：</p>

<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nc">Binding</span> <span class="n">existing</span> <span class="o">=</span> <span class="n">findBySubjectAndTarget</span><span class="o">(</span><span class="n">subject</span><span class="o">,</span> <span class="n">target</span><span class="o">);</span>
<span class="k">if</span> <span class="o">(</span><span class="n">existing</span> <span class="o">==</span> <span class="kc">null</span><span class="o">)</span> <span class="o">{</span>
    <span class="n">insert</span><span class="o">(</span><span class="n">newRow</span><span class="o">);</span>                                     <span class="c1">// 首次申请</span>
<span class="o">}</span> <span class="k">else</span> <span class="k">if</span> <span class="o">(</span><span class="no">REAPPLIABLE</span><span class="o">.</span><span class="na">contains</span><span class="o">(</span><span class="n">existing</span><span class="o">.</span><span class="na">status</span><span class="o">()))</span> <span class="o">{</span>
    <span class="kt">int</span> <span class="n">n</span> <span class="o">=</span> <span class="n">update</span><span class="o">(</span>
        <span class="s">"UPDATE binding SET status = ?, operation_id = ?, expires_at = ? "</span> <span class="o">+</span>
        <span class="s">"WHERE subject_id = ? AND target_key = ? AND status = ?"</span><span class="o">,</span>   <span class="c1">// 锁定读到的确切旧状态</span>
        <span class="n">newRow</span><span class="o">.</span><span class="na">status</span><span class="o">(),</span> <span class="n">newRow</span><span class="o">.</span><span class="na">operationId</span><span class="o">(),</span> <span class="n">newRow</span><span class="o">.</span><span class="na">expiresAt</span><span class="o">(),</span>
        <span class="n">subject</span><span class="o">,</span> <span class="n">target</span><span class="o">,</span> <span class="n">existing</span><span class="o">.</span><span class="na">status</span><span class="o">());</span>
    <span class="k">if</span> <span class="o">(</span><span class="n">n</span> <span class="o">==</span> <span class="mi">0</span><span class="o">)</span> <span class="k">throw</span> <span class="k">new</span> <span class="nc">PreemptedException</span><span class="o">();</span>         <span class="c1">// 并发的重新申请或状态变化</span>
<span class="o">}</span> <span class="k">else</span> <span class="o">{</span>
    <span class="k">throw</span> <span class="k">new</span> <span class="nf">ConstraintViolationException</span><span class="o">();</span>           <span class="c1">// 拒绝覆盖活跃或待恢复的行</span>
<span class="o">}</span>
</code></pre></div></div>

<ul>
  <li><strong>只有”已拒绝”“已关闭”等可重新申请的终态才允许原地重置。</strong> 重置会替换 <code class="language-plaintext highlighter-rouge">operation_id</code>，这正是上一篇里过期回调被识别出来的依据。</li>
  <li><strong>其他状态一律拒绝覆盖。</strong> 如果覆盖一个仍待人工恢复的失败行，它的 <code class="language-plaintext highlighter-rouge">operation_id</code> 会被替换，挂在旧 <code class="language-plaintext highlighter-rouge">operation_id</code> 上的恢复流程将找不到目标。</li>
  <li><strong>WHERE 条件锁定读到的确切旧状态</strong>，而不是”任一可重新申请状态”，从而在多个状态都可重新申请时，仍能发现读写之间发生的变化。</li>
  <li>每次申请的历史保存在独立的历史表中，主表只保存当前状态。</li>
</ul>

<h3 id="36-跨表转移固定加锁顺序只有真正翻转时才返回结果">3.6 跨表转移：固定加锁顺序，只有真正翻转时才返回结果</h3>

<p>需要同时修改 <code class="language-plaintext highlighter-rouge">operation</code> 和 <code class="language-plaintext highlighter-rouge">binding</code> 两张表的转移，统一先锁 <code class="language-plaintext highlighter-rouge">operation</code>、再锁 <code class="language-plaintext highlighter-rouge">binding</code>，不同路径使用相同的加锁顺序，避免死锁。推进方法只在本次调用真正改变了状态时返回非空结果，调用方据此决定是否产生副作用（发送指标、通知）。这样并发的两条路径中，只有一条会产生副作用（细节见上一篇 3.7 节）。</p>

<h3 id="37-批次失败结算先关闭再重读以已合并为准">3.7 批次失败结算：先关闭再重读，以已合并为准</h3>

<p>批次失败时 MR 可能已经存在，并且可能恰好在失败判定之后合并成功。结算流程是：如果 MR 仍处于打开状态，先关闭它，然后重新读取 MR 的最终状态。</p>

<ul>
  <li><strong>读到 merged</strong>：按成功处理，把所有成员对齐到”已合并”，批次标记为 SUCCEEDED。已合并的 MR 无法再被关闭，所以”合并”与”关闭”竞争时，合并的结果不会丢失。</li>
  <li><strong>读到 closed 或不存在 MR</strong>：批次标记为 FAILED，在同一个事务中删除各成员的处理记录；只追加的成员历史和审计记录保留。</li>
</ul>

<h2 id="4-成果">4. 成果</h2>

<ul>
  <li><strong>六条不变量都由数据库约束或守卫式写入保证。</strong> 并发认领、重复入队、重复请求、过期的状态写入，都会在数据库层面被拒绝或被识别，不依赖调用时序。</li>
  <li><strong>没有引入新组件。</strong> 单活互斥没有锁表、租约或外部锁服务，互斥状态与业务状态是同一列，不存在两者不一致的情况。</li>
  <li><strong>入队不被批次阻塞。</strong> 认领事务不对等待区间加锁，批次运行期间新申请照常写入。</li>
  <li><strong>崩溃后状态显式。</strong> 崩溃的批次停在可查询的 RUNNING 状态，由超时扫描或运维工具结算，不存在看不见的泄漏锁。</li>
  <li><strong>定时触发与手动触发可以安全重叠。</strong> 两者执行同一段代码，重叠时后到的一方得到”槽位被占用”，不需要额外协调。</li>
</ul>

<h2 id="5-已知限制与后续计划">5. 已知限制与后续计划</h2>

<ul>
  <li><strong>状态即版本存在 ABA 问题。</strong> 一行在读写之间离开某个状态又回到该状态，守卫条件仍会通过。表中预留了 <code class="language-plaintext highlighter-rouge">version</code> 列，目前尚未启用。对于可能往返的状态，后续计划改用单调递增的版本号。</li>
  <li><strong>全局只有一个槽位。</strong> 当前设计不支持按维度并行。如果需要按分区并行，可以把生成列改为 <code class="language-plaintext highlighter-rouge">CASE WHEN status IN (1, 4) THEN partition_key END</code>，唯一索引就变成”每个分区最多一个活跃批次”。</li>
  <li><strong>批量路径存在逐行访问。</strong> 移交阶段按申请逐条回查关系行（N+1）；成员历史逐行插入；<code class="language-plaintext highlighter-rouge">IN</code> 列表最多可达上千个占位符，且未分片。后续计划改为批量查询、批量插入，并对 <code class="language-plaintext highlighter-rouge">IN</code> 列表分片。</li>
  <li><strong>部分查询缺少合适的索引。</strong> 对账查询按 <code class="language-plaintext highlighter-rouge">status</code> 加时间范围过滤，但只有单列时间索引；取最大 MR 编号使用 <code class="language-plaintext highlighter-rouge">ORDER BY CAST(mr_id AS UNSIGNED)</code>，表达式排序无法利用索引。后续计划补充 <code class="language-plaintext highlighter-rouge">(status, updated_at)</code> 复合索引，并把 MR 编号改存为数值列。</li>
  <li><strong>幂等记录的过期时间未生效。</strong> 记录带有过期时间字段，但没有清理任务，也没有对应的索引。</li>
  <li><strong>时间精度为秒。</strong> <code class="language-plaintext highlighter-rouge">created_at</code> 精度为秒，认领条件是 <code class="language-plaintext highlighter-rouge">created_at &lt; CURRENT_TIMESTAMP</code>，因此当前这一秒内入队的申请要等到下一轮。这是有意的取舍，但需要在文档中写明。</li>
</ul>]]></content><author><name></name></author><category term="zh" /><summary type="html"><![CDATA[不引入分布式锁，用生成列加唯一索引、守卫式 UPDATE 和唯一约束，让 MySQL 直接保证批量聚合的六条业务不变量。]]></summary></entry><entry xml:lang="zh-CN"><title type="html">用 Merge Request 驱动长流程：状态机、幂等与对账的工程实践</title><link href="https://blog.changcodes.com/zh/posts/mr-driven-long-workflow/" rel="alternate" type="text/html" title="用 Merge Request 驱动长流程：状态机、幂等与对账的工程实践" /><published>2026-03-15T00:00:00+00:00</published><updated>2026-03-15T00:00:00+00:00</updated><id>https://blog.changcodes.com/zh/posts/mr-driven-long-workflow</id><content type="html" xml:base="https://blog.changcodes.com/zh/posts/mr-driven-long-workflow/"><![CDATA[<h2 id="1-问题与目标">1. 问题与目标</h2>

<p>我们维护一个”配置即代码”的自助变更系统。用户提交一次变更申请后，系统需要依次完成：</p>

<ol>
  <li>向外部审批系统发起审批，等待审批结果回调；</li>
  <li>以机器人账号在 Git 仓库创建分支、提交文件、发起 Merge Request（下称 MR）；</li>
  <li>等待 MR 的检查流水线通过；</li>
  <li>以审批机器人身份批准 MR 并合并，合并失败时 rebase 后重试；</li>
  <li>等待部署流水线把配置下发，并通过回调告知系统”已生效”。</li>
</ol>

<p>一次变更跨越审批系统、Git 托管平台（GitLab）和 CI 三个外部系统，耗时从几分钟到几小时不等。如果把它当作一次普通的同步请求来处理，会面临几个问题：HTTP 连接无法等待这么久；任何一步失败都可能让请求停在一个无人知晓的中间态；外部系统的回调可能重复、迟到或乱序；进程重启会丢失正在执行的步骤。</p>

<p>我们为这条流程设定了四个目标：</p>

<ul>
  <li><strong>可观察</strong>：任何时刻，一个请求都处在一个可解释的状态，并且能看出它停在哪一步。</li>
  <li><strong>失败可见</strong>：失败必须落成一个明确的失败状态，不能表现为永远”处理中”。</li>
  <li><strong>对事件鲁棒</strong>：重复、迟到、属于旧申请的事件不得改变当前申请的状态。</li>
  <li><strong>可演进</strong>：调整流程顺序、适配上游接口变化时，不中断在途请求。</li>
</ul>

<h2 id="2-约束与考量">2. 约束与考量</h2>

<p><strong>外部调用不在本地事务内。</strong> 在 GitLab 上创建分支、提交和 MR 之后，本地写库仍可能失败；反过来，本地状态已经写入，远端调用也可能失败。两边之间不存在可以依赖的原子性。</p>

<p><strong>事件来源多，且投递语义弱。</strong> 驱动状态变化的来源包括：用户提交、审批系统回调、系统自身推进、部署流水线回调、定时对账、运维人员手动恢复。外部回调按”至少一次”投递，可能迟到，也可能早于我们自己的状态写入到达。</p>

<p><strong>上游响应存在歧义。</strong> GitLab 对”同一账号重复批准”返回 HTTP 401，与”该账号无权批准”是同一个状态码。只看响应无法区分两者。</p>

<p><strong>部署形态。</strong> 服务多实例部署。我们不希望为这一条流程引入工作流引擎或分布式锁等新组件，现有的共享协调点只有关系数据库。</p>

<p><strong>流程需要演进。</strong> 流程顺序要从”先建 MR 再审批”调整为”先审批再建 MR”，切换时已有审批在途。上游平台还计划在审批接口中新增一个必填的结构化字段，并给出了兼容期。</p>

<h2 id="3-方案">3. 方案</h2>

<h3 id="31-两层状态步骤状态与对外状态">3.1 两层状态：步骤状态与对外状态</h3>

<p>我们用两张表分别记录两个粒度的状态：</p>

<ul>
  <li><code class="language-plaintext highlighter-rouge">operation</code> 表记录<strong>步骤状态</strong>，细到”流水线超时”“rebase 冲突”“机器人批准失败”。状态码按阶段划分号段（创建阶段 10001 起、处理阶段 10101 起、关闭阶段 10201 起），按号段前缀即可聚合统计。</li>
  <li><code class="language-plaintext highlighter-rouge">binding</code> 表记录<strong>对外状态</strong>，只表达用户关心的阶段：审批中、MR 创建中、合并中、已合并、已生效、失败、已拒绝。该表以 <code class="language-plaintext highlighter-rouge">(subject_id, target_key)</code> 为唯一键，每个”主体—目标”关系只有一行。</li>
</ul>

<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">enum</span> <span class="nc">StepStatus</span> <span class="o">{</span>            <span class="c1">// 细粒度：服务于排障和指标</span>
    <span class="no">CREATE_STARTED</span><span class="o">(</span><span class="mi">10001</span><span class="o">),</span> <span class="no">CREATE_SUCCEEDED</span><span class="o">(</span><span class="mi">10002</span><span class="o">),</span> <span class="no">CREATE_PIPELINE_FAILED</span><span class="o">(</span><span class="mi">10004</span><span class="o">),</span>
    <span class="no">HANDLE_STARTED</span><span class="o">(</span><span class="mi">10101</span><span class="o">),</span> <span class="no">HANDLE_APPROVED</span><span class="o">(</span><span class="mi">10102</span><span class="o">),</span> <span class="no">HANDLE_REBASE_CONFLICT</span><span class="o">(</span><span class="mi">10105</span><span class="o">),</span>
    <span class="no">HANDLE_PIPELINE_PENDING</span><span class="o">(</span><span class="mi">10501</span><span class="o">),</span> <span class="no">CALLED_BACK</span><span class="o">(</span><span class="mi">10301</span><span class="o">),</span> <span class="cm">/* ... */</span><span class="o">;</span>
<span class="o">}</span>

<span class="kd">enum</span> <span class="nc">BindingStatus</span> <span class="o">{</span>         <span class="c1">// 粗粒度：服务于展示和"能否重新申请"</span>
    <span class="no">APPROVAL_SENDING</span><span class="o">,</span> <span class="no">APPROVAL_SENT</span><span class="o">,</span> <span class="no">MR_CREATING</span><span class="o">,</span> <span class="no">MR_CREATED</span><span class="o">,</span> <span class="no">MERGING</span><span class="o">,</span> <span class="no">MERGED</span><span class="o">,</span> <span class="no">VALIDATED</span><span class="o">,</span>
    <span class="no">CREATE_FAILED</span><span class="o">,</span> <span class="no">HANDLE_FAILED</span><span class="o">,</span> <span class="no">REJECTED</span><span class="o">,</span> <span class="no">CLOSED</span><span class="o">;</span>

    <span class="cm">/** 唯一的事实来源：数据层和接口层都引用它判断能否重新申请。 */</span>
    <span class="kd">static</span> <span class="kd">final</span> <span class="nc">Set</span><span class="o">&lt;</span><span class="nc">BindingStatus</span><span class="o">&gt;</span> <span class="no">REAPPLIABLE</span> <span class="o">=</span> <span class="nc">Set</span><span class="o">.</span><span class="na">of</span><span class="o">(</span><span class="no">REJECTED</span><span class="o">,</span> <span class="no">CLOSED</span><span class="o">);</span>
<span class="o">}</span>
</code></pre></div></div>

<p>之所以拆成两层，是因为两类读者的需求不同。排障需要知道请求停在哪个具体步骤；用户和”能否重新申请”的判断只关心所处的大阶段。如果只用一个状态字段，每次细化内部步骤都会牵动前端展示和重新申请规则。</p>

<p>每次状态转移都在同一个本地事务里同时写两张表：</p>

<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kt">void</span> <span class="nf">transition</span><span class="o">(</span><span class="nc">OperationBuilder</span> <span class="n">op</span><span class="o">,</span> <span class="nc">StepStatus</span> <span class="n">step</span><span class="o">,</span> <span class="nc">BindingStatus</span> <span class="n">status</span><span class="o">)</span> <span class="o">{</span>
    <span class="n">tx</span><span class="o">.</span><span class="na">execute</span><span class="o">(</span><span class="nc">List</span><span class="o">.</span><span class="na">of</span><span class="o">(</span>
        <span class="o">()</span> <span class="o">-&gt;</span> <span class="n">operationRepo</span><span class="o">.</span><span class="na">update</span><span class="o">(</span><span class="n">op</span><span class="o">.</span><span class="na">status</span><span class="o">(</span><span class="n">step</span><span class="o">)),</span>          <span class="c1">// 以刚读到的状态做乐观并发校验</span>
        <span class="o">()</span> <span class="o">-&gt;</span> <span class="n">bindingRepo</span><span class="o">.</span><span class="na">updateStatus</span><span class="o">(</span><span class="n">op</span><span class="o">.</span><span class="na">id</span><span class="o">(),</span> <span class="n">status</span><span class="o">)</span>
    <span class="o">));</span>
<span class="o">}</span>
</code></pre></div></div>

<p>下图是对外状态的主路径（省略了各类失败分支）：</p>

<pre><code class="language-mermaid">stateDiagram-v2
    [*] --&gt; APPROVAL_SENT: 提交申请
    APPROVAL_SENT --&gt; REJECTED: 审批拒绝或撤回
    APPROVAL_SENT --&gt; MR_CREATING: 审批通过
    MR_CREATING --&gt; MR_CREATED: 检查流水线通过
    MR_CREATING --&gt; CREATE_FAILED
    MR_CREATED --&gt; MERGING
    MERGING --&gt; MERGED: 合并成功
    MERGING --&gt; HANDLE_FAILED
    MERGED --&gt; VALIDATED: 部署回调或定时对账
    REJECTED --&gt; [*]
    VALIDATED --&gt; [*]
</code></pre>

<h3 id="32-入口异步化失败一律落库">3.2 入口异步化，失败一律落库</h3>

<p>提交接口只做同步校验（参数、幂等、是否已有进行中的申请），然后把长流程提交到有界线程池，立即返回 HTTP 202 和 <code class="language-plaintext highlighter-rouge">operationId</code>。客户端按 <code class="language-plaintext highlighter-rouge">operationId</code> 轮询。</p>

<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">private</span> <span class="kt">void</span> <span class="nf">fireAsync</span><span class="o">(</span><span class="nc">String</span> <span class="n">opId</span><span class="o">,</span> <span class="nc">Task</span> <span class="n">task</span><span class="o">,</span> <span class="nc">StepStatus</span> <span class="n">failStep</span><span class="o">,</span> <span class="nc">BindingStatus</span> <span class="n">failStatus</span><span class="o">)</span> <span class="o">{</span>
    <span class="k">try</span> <span class="o">{</span>
        <span class="n">executor</span><span class="o">.</span><span class="na">execute</span><span class="o">(()</span> <span class="o">-&gt;</span> <span class="o">{</span>
            <span class="k">try</span> <span class="o">{</span>
                <span class="n">task</span><span class="o">.</span><span class="na">run</span><span class="o">();</span>
            <span class="o">}</span> <span class="k">catch</span> <span class="o">(</span><span class="nc">Exception</span> <span class="n">e</span><span class="o">)</span> <span class="o">{</span>
                <span class="n">log</span><span class="o">.</span><span class="na">error</span><span class="o">(</span><span class="s">"async step failed"</span><span class="o">,</span> <span class="n">e</span><span class="o">);</span>        <span class="c1">// 失败状态已由切面写入，见下文</span>
            <span class="o">}</span> <span class="k">catch</span> <span class="o">(</span><span class="nc">Throwable</span> <span class="n">t</span><span class="o">)</span> <span class="o">{</span>
                <span class="n">log</span><span class="o">.</span><span class="na">error</span><span class="o">(</span><span class="s">"[CRITICAL] JVM-level error"</span><span class="o">,</span> <span class="n">t</span><span class="o">);</span> <span class="c1">// JVM 可能已处于异常状态，只记录不写库</span>
            <span class="o">}</span>
        <span class="o">});</span>
    <span class="o">}</span> <span class="k">catch</span> <span class="o">(</span><span class="nc">RejectedExecutionException</span> <span class="n">e</span><span class="o">)</span> <span class="o">{</span>
        <span class="c1">// 线程池满或正在关闭：把失败写进库，轮询方看到"失败"而不是"处理中"</span>
        <span class="n">tx</span><span class="o">.</span><span class="na">execute</span><span class="o">(</span><span class="nc">List</span><span class="o">.</span><span class="na">of</span><span class="o">(</span>
            <span class="o">()</span> <span class="o">-&gt;</span> <span class="n">operationRepo</span><span class="o">.</span><span class="na">updateStatus</span><span class="o">(</span><span class="n">opId</span><span class="o">,</span> <span class="n">failStep</span><span class="o">),</span>
            <span class="o">()</span> <span class="o">-&gt;</span> <span class="n">bindingRepo</span><span class="o">.</span><span class="na">updateStatus</span><span class="o">(</span><span class="n">opId</span><span class="o">,</span> <span class="n">failStatus</span><span class="o">)));</span>
    <span class="o">}</span>
<span class="o">}</span>
</code></pre></div></div>

<p>线程池的核心线程数等于最大线程数，队列有界，拒绝策略选择 <code class="language-plaintext highlighter-rouge">AbortPolicy</code>。没有选另外两种常见做法，原因如下：</p>

<ul>
  <li><code class="language-plaintext highlighter-rouge">CallerRunsPolicy</code> 会让 Web 线程执行一个耗时数分钟的任务，过载会被传导到所有接口。</li>
  <li>无界队列会掩盖过载，任务可能在队列里等很久，期间用户看到的始终是”处理中”。</li>
</ul>

<p><code class="language-plaintext highlighter-rouge">AbortPolicy</code> 加上”拒绝即落库”，把过载转化成一个用户可见、可以统计的失败状态。</p>

<p>每个步骤方法通过注解声明自己的失败状态，由切面统一兜底：</p>

<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nd">@WorkflowStep</span><span class="o">(</span><span class="n">onErrorStep</span> <span class="o">=</span> <span class="nc">StepStatus</span><span class="o">.</span><span class="na">HANDLE_MERGE_FAILED</span><span class="o">,</span>
              <span class="n">onErrorStatus</span> <span class="o">=</span> <span class="nc">BindingStatus</span><span class="o">.</span><span class="na">HANDLE_FAILED</span><span class="o">)</span>
<span class="kd">public</span> <span class="nc">Result</span> <span class="nf">handleMergeRequest</span><span class="o">(</span><span class="nc">String</span> <span class="n">opId</span><span class="o">,</span> <span class="kt">long</span> <span class="n">mrIid</span><span class="o">)</span> <span class="o">{</span> <span class="o">...</span> <span class="o">}</span>
</code></pre></div></div>

<p>方法内部已经识别的失败（如流水线失败、rebase 冲突）由业务代码写入更具体的状态；未预料的异常由切面写入注解声明的状态。两者合起来，保证任何一条退出路径都会留下终态。</p>

<h3 id="33-回调守卫只接受当前所有者且正在等待的事件">3.3 回调守卫：只接受”当前所有者”且”正在等待”的事件</h3>

<p>审批回调的处理分两层。第一层按回调自身携带的请求 ID 做幂等，挡住重复投递。第二层判断这个回调是否仍然有效：</p>

<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nc">Binding</span> <span class="n">row</span> <span class="o">=</span> <span class="n">bindingRepo</span><span class="o">.</span><span class="na">findByOperationId</span><span class="o">(</span><span class="n">callback</span><span class="o">.</span><span class="na">operationId</span><span class="o">());</span>
<span class="k">if</span> <span class="o">(</span><span class="n">row</span> <span class="o">==</span> <span class="kc">null</span> <span class="o">||</span> <span class="o">!</span><span class="no">AWAITING_APPROVAL</span><span class="o">.</span><span class="na">contains</span><span class="o">(</span><span class="n">row</span><span class="o">.</span><span class="na">status</span><span class="o">()))</span> <span class="o">{</span>
    <span class="k">return</span> <span class="nf">accepted</span><span class="o">(</span><span class="s">"IGNORED_STALE"</span><span class="o">);</span>   <span class="c1">// 返回 2xx，避免上游把它当作失败反复重试</span>
<span class="o">}</span>
</code></pre></div></div>

<p>这里有两个设计点。</p>

<ul>
  <li><strong>按 operationId 判断所有权。</strong> 用户重新申请时，<code class="language-plaintext highlighter-rouge">binding</code> 行会被原地重置并换上新的 <code class="language-plaintext highlighter-rouge">operationId</code>（见数据库篇）。旧申请的迟到回调因此查不到行，被直接忽略，不会把新申请推进到错误的状态。</li>
  <li><strong>等待集合包含”发送中”。</strong> <code class="language-plaintext highlighter-rouge">AWAITING_APPROVAL = {APPROVAL_SENDING, APPROVAL_SENT}</code>。我们在调用审批接口之前写入 SENDING，调用成功后写入 SENT。审批系统的回调可能比 SENT 的写入更早到达，如果等待集合只包含 SENT，这类回调会被误判为过期。</li>
</ul>

<h3 id="34-以远端状态为准的幂等">3.4 以远端状态为准的幂等</h3>

<p>本地状态与远端状态之间没有原子性，所以每个会改动远端的步骤在执行前先读远端状态：</p>

<ul>
  <li>处理 MR 之前，如果 MR 已经是 merged，直接按”已合并”收尾，不再批准。否则重新批准会被 GitLab 拒绝，记录下来的会是”批准失败”，把排查方向引向权限问题。</li>
  <li>关闭 MR 之前，如果它已经是 closed 或 merged，跳过关闭调用。</li>
</ul>

<p>机器人批准这一步需要额外处理 401 的歧义：</p>

<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kt">void</span> <span class="nf">approveAsReviewer</span><span class="o">(</span><span class="nc">Api</span> <span class="n">reviewer</span><span class="o">,</span> <span class="kt">long</span> <span class="n">mrIid</span><span class="o">)</span> <span class="kd">throws</span> <span class="nc">ApiException</span><span class="o">,</span> <span class="nc">InterruptedException</span> <span class="o">{</span>
    <span class="kt">int</span> <span class="n">maxRetries</span> <span class="o">=</span> <span class="nc">Math</span><span class="o">.</span><span class="na">max</span><span class="o">(</span><span class="mi">0</span><span class="o">,</span> <span class="n">config</span><span class="o">.</span><span class="na">getInt</span><span class="o">(</span><span class="s">"approval.maxRetries"</span><span class="o">,</span> <span class="mi">3</span><span class="o">));</span> <span class="c1">// 负值按 0 处理</span>
    <span class="nc">ApiException</span> <span class="n">last</span> <span class="o">=</span> <span class="kc">null</span><span class="o">;</span>
    <span class="k">for</span> <span class="o">(</span><span class="kt">int</span> <span class="n">attempt</span> <span class="o">=</span> <span class="mi">1</span><span class="o">;</span> <span class="n">attempt</span> <span class="o">&lt;=</span> <span class="n">maxRetries</span> <span class="o">+</span> <span class="mi">1</span><span class="o">;</span> <span class="n">attempt</span><span class="o">++)</span> <span class="o">{</span>
        <span class="k">if</span> <span class="o">(</span><span class="n">hasApprovalFrom</span><span class="o">(</span><span class="n">reviewer</span><span class="o">,</span> <span class="n">mrIid</span><span class="o">))</span> <span class="o">{</span>
            <span class="k">return</span><span class="o">;</span>                                   <span class="c1">// 已经批准过：跳过调用</span>
        <span class="o">}</span>
        <span class="k">try</span> <span class="o">{</span>
            <span class="n">approver</span><span class="o">.</span><span class="na">approve</span><span class="o">(</span><span class="n">reviewer</span><span class="o">,</span> <span class="n">mrIid</span><span class="o">);</span>
            <span class="k">return</span><span class="o">;</span>
        <span class="o">}</span> <span class="k">catch</span> <span class="o">(</span><span class="nc">ApiException</span> <span class="n">e</span><span class="o">)</span> <span class="o">{</span>
            <span class="n">last</span> <span class="o">=</span> <span class="n">e</span><span class="o">;</span>
            <span class="k">if</span> <span class="o">(</span><span class="n">attempt</span> <span class="o">&gt;</span> <span class="n">maxRetries</span><span class="o">)</span> <span class="k">break</span><span class="o">;</span>
            <span class="nc">Thread</span><span class="o">.</span><span class="na">sleep</span><span class="o">(</span><span class="n">backoffBaseMs</span> <span class="o">&lt;&lt;</span> <span class="o">(</span><span class="n">attempt</span> <span class="o">-</span> <span class="mi">1</span><span class="o">));</span> <span class="c1">// 指数退避</span>
        <span class="o">}</span>
    <span class="o">}</span>
    <span class="c1">// 重复批准与无权批准都返回 401。最后再读一次，确认批准是否其实已经存在。</span>
    <span class="k">if</span> <span class="o">(</span><span class="n">hasApprovalFrom</span><span class="o">(</span><span class="n">reviewer</span><span class="o">,</span> <span class="n">mrIid</span><span class="o">))</span> <span class="o">{</span>
        <span class="k">return</span><span class="o">;</span>
    <span class="o">}</span>
    <span class="k">throw</span> <span class="n">last</span><span class="o">;</span>
<span class="o">}</span>
</code></pre></div></div>

<ul>
  <li><strong>每次尝试前读一次批准人列表。</strong> 如果上一次批准请求已到达 GitLab 但响应丢失，下一次尝试会看到批准已存在，而不是再收到一个 401。</li>
  <li><strong>最终失败后再读一次。</strong> 最后一次尝试之后没有下一次尝试来纠正，这次读取覆盖了这个情况。</li>
  <li><strong>读取是 fail-open 的。</strong> 读不到批准人列表时，按”尚未批准”处理，继续尝试批准。这保持了引入该检查之前的行为，检查本身不会成为新的失败来源。</li>
  <li><strong>两个审批账号分别重试。</strong> 两次批准不是一个原子操作，如果把它们作为一对整体重试，会对第一个账号重复提交批准。</li>
</ul>

<p>这些检查让”从中断处重新执行”变得安全，运维恢复工具可以直接重新触发某个步骤。</p>

<h3 id="35-等待外部状态阶梯式轮询区分超时与失败">3.5 等待外部状态：阶梯式轮询，区分”超时”与”失败”</h3>

<p>rebase 结果和流水线状态都需要轮询。我们把轮询抽象成一个通用组件：前 N 次用短间隔，之后改用长间隔，达到上限后返回超时。轮询结果有三种：通过、失败、超时。</p>

<ul>
  <li><strong>流水线失败</strong>是确定性的，用同一个 MR 重试仍会失败，因此进入失败状态。</li>
  <li><strong>流水线超时</strong>表示还不知道结果，对外状态保持在”合并中”，步骤状态记为”流水线待定”，交由人工确认。</li>
</ul>

<p>接口层据此向用户给出不同的提示：确定性失败提示”需要联系维护团队”，结果未知的情况提示”可以继续等待”（详见 API 篇）。</p>

<h3 id="36-在线调整流程顺序无需排空">3.6 在线调整流程顺序，无需排空</h3>

<p>把流程从”先建 MR 再审批”改为”先审批再建 MR”时，我们没有设置停机或排空窗口，而是让两种顺序在一段时间内并存：</p>

<ol>
  <li><strong>新流程由配置开关控制</strong>，默认关闭，可以随时切回旧流程。</li>
  <li>
    <p><strong>回调按载荷形态区分流程。</strong> 旧流程在发起审批时 MR 已经存在，会把 MR 编号放进审批单的扩展字段；新流程发起审批时还没有 MR。回调处理据此判断：</p>

    <div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kt">boolean</span> <span class="n">legacyFlow</span> <span class="o">=</span> <span class="nc">StringUtils</span><span class="o">.</span><span class="na">isNotEmpty</span><span class="o">(</span><span class="n">callback</span><span class="o">.</span><span class="na">ext</span><span class="o">().</span><span class="na">get</span><span class="o">(</span><span class="s">"mrId"</span><span class="o">));</span>
<span class="k">if</span> <span class="o">(</span><span class="n">approved</span><span class="o">)</span> <span class="o">{</span>
    <span class="k">if</span> <span class="o">(</span><span class="n">legacyFlow</span><span class="o">)</span> <span class="n">workflow</span><span class="o">.</span><span class="na">fireHandle</span><span class="o">(</span><span class="n">opId</span><span class="o">,</span> <span class="n">mrId</span><span class="o">);</span>      <span class="c1">// MR 已存在：批准并合并</span>
    <span class="k">else</span>            <span class="n">workflow</span><span class="o">.</span><span class="na">fireCreateAfterApproval</span><span class="o">(</span><span class="n">opId</span><span class="o">);</span> <span class="c1">// 先创建 MR</span>
<span class="o">}</span>
</code></pre></div>    </div>

    <p>切换开关前发出、切换后才回调的审批，仍按旧流程处理。</p>
  </li>
  <li>
    <p><strong>创建步骤通过查库判断”插入还是推进”</strong>，而不是依赖调用方传入的标志：</p>

    <div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">if</span> <span class="o">(</span><span class="n">operationRepo</span><span class="o">.</span><span class="na">exists</span><span class="o">(</span><span class="n">opId</span><span class="o">))</span> <span class="o">{</span>
    <span class="n">transition</span><span class="o">(</span><span class="n">op</span><span class="o">,</span> <span class="no">CREATE_STARTED</span><span class="o">,</span> <span class="no">MR_CREATING</span><span class="o">);</span>   <span class="c1">// 新流程：行在发起审批时已创建</span>
<span class="o">}</span> <span class="k">else</span> <span class="o">{</span>
    <span class="n">insertOperationAndBinding</span><span class="o">(</span><span class="n">op</span><span class="o">);</span>                 <span class="c1">// 旧流程：首次创建</span>
<span class="o">}</span>
</code></pre></div>    </div>

    <p>这让同一个步骤对两种顺序都成立，对运维重新触发也保持幂等。</p>
  </li>
</ol>

<h3 id="37-丢失回调的定时对账">3.7 丢失回调的定时对账</h3>

<p>部署流水线的回调可能丢失，相应的请求会停在”已合并”。我们用一个定时任务收敛这类记录。</p>

<p><strong>滑动时间窗口。</strong> 每次只扫描最后变更时间位于 <code class="language-plaintext highlighter-rouge">(now - maxAge, now - minAge)</code> 内的”已合并”记录。早于 <code class="language-plaintext highlighter-rouge">minAge</code> 的记录可能只是回调还在路上，跳过；晚于 <code class="language-plaintext highlighter-rouge">maxAge</code> 的记录移出窗口，以此限制单次工作量。</p>

<p><strong>用水位线代替逐行查询。</strong> 回调表只记录 MR 编号，不含业务字段。对账任务只做两次查询：一次 <code class="language-plaintext highlighter-rouge">IN</code> 查询批量取出 <code class="language-plaintext highlighter-rouge">operationId → mrIid</code>，一次查询取回调表中最大的 MR 编号。之后对每一行做一次比较：</p>

<div class="language-text highlighter-rouge"><div class="highlight"><pre class="highlight"><code>mrIid &lt;= maxCallbackIid  → 推进为"已生效"
mrIid &gt;  maxCallbackIid  → 保持"已合并"，等待下一轮
</code></pre></div></div>

<p><code class="language-plaintext highlighter-rouge">mrIid == max</code> 表示这个 MR 自己的回调已经到达；<code class="language-plaintext highlighter-rouge">mrIid &lt; max</code> 表示编号更大的 MR 已有回调，而 MR 编号在项目内单调递增，由此推断较早的这条回调已经丢失。这条推断成立的前提是：部署流水线按合并顺序执行，且后一次执行不会遗漏之前已合并的变更。<code class="language-plaintext highlighter-rouge">minAge</code> 窗口用来防止把”迟到”误判为”丢失”。</p>

<p><strong>与正常回调路径并发时只生效一次。</strong> 回调处理和对账任务都会调用同一个状态推进方法：</p>

<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nd">@Transactional</span>
<span class="kd">public</span> <span class="nc">String</span> <span class="nf">markValidated</span><span class="o">(</span><span class="nc">String</span> <span class="n">opId</span><span class="o">)</span> <span class="o">{</span>
    <span class="nc">Operation</span> <span class="n">op</span> <span class="o">=</span> <span class="n">operationRepo</span><span class="o">.</span><span class="na">selectForUpdate</span><span class="o">(</span><span class="n">opId</span><span class="o">);</span>  <span class="c1">// 固定加锁顺序：先 operation</span>
    <span class="nc">Binding</span>   <span class="n">b</span>  <span class="o">=</span> <span class="n">bindingRepo</span><span class="o">.</span><span class="na">selectForUpdate</span><span class="o">(</span><span class="n">opId</span><span class="o">);</span>    <span class="c1">// 再 binding</span>
    <span class="k">if</span> <span class="o">(</span><span class="n">b</span> <span class="o">==</span> <span class="kc">null</span> <span class="o">||</span> <span class="o">(</span><span class="n">b</span><span class="o">.</span><span class="na">status</span><span class="o">()</span> <span class="o">!=</span> <span class="no">MERGED</span> <span class="o">&amp;&amp;</span> <span class="n">b</span><span class="o">.</span><span class="na">status</span><span class="o">()</span> <span class="o">!=</span> <span class="no">VALIDATED</span><span class="o">))</span> <span class="o">{</span>
        <span class="k">return</span> <span class="kc">null</span><span class="o">;</span>                                     <span class="c1">// 不符合条件：幂等空操作</span>
    <span class="o">}</span>
    <span class="k">if</span> <span class="o">(</span><span class="n">op</span><span class="o">.</span><span class="na">status</span><span class="o">()</span> <span class="o">!=</span> <span class="no">CALLED_BACK</span><span class="o">)</span> <span class="o">{</span>
        <span class="n">operationRepo</span><span class="o">.</span><span class="na">updateStatus</span><span class="o">(</span><span class="n">opId</span><span class="o">,</span> <span class="no">CALLED_BACK</span><span class="o">);</span>   <span class="c1">// 修复可能滞后的步骤状态</span>
    <span class="o">}</span>
    <span class="k">if</span> <span class="o">(</span><span class="n">b</span><span class="o">.</span><span class="na">status</span><span class="o">()</span> <span class="o">==</span> <span class="no">MERGED</span><span class="o">)</span> <span class="o">{</span>
        <span class="n">requireOneRow</span><span class="o">(</span><span class="n">bindingRepo</span><span class="o">.</span><span class="na">update</span><span class="o">(</span>
            <span class="s">"SET status = VALIDATED WHERE operation_id = ? AND status = MERGED"</span><span class="o">,</span> <span class="n">opId</span><span class="o">));</span>
        <span class="k">return</span> <span class="n">b</span><span class="o">.</span><span class="na">subjectId</span><span class="o">();</span>                            <span class="c1">// 只有真正翻转时才返回非空</span>
    <span class="o">}</span>
    <span class="k">return</span> <span class="kc">null</span><span class="o">;</span>
<span class="o">}</span>
</code></pre></div></div>

<p>两条路径并发时，行锁保证只有一方完成翻转，另一方读到 VALIDATED 后返回 <code class="language-plaintext highlighter-rouge">null</code>。调用方只在返回非空时计入”生效成功”指标，并且等外层事务提交之后再统一发出，所以事务回滚不会留下多计的指标，并发也不会导致重复计数。</p>

<h3 id="38-适配上游接口变更">3.8 适配上游接口变更</h3>

<p>上游平台计划在批准接口中新增一个必填字段：一组需要逐项确认的检查项。我们使用的 GitLab 客户端库无法表达这个字段。处理方式如下：</p>

<ul>
  <li><strong>把批准动作收敛到一个独立组件。</strong> 组件内有两种实现：沿用客户端库的旧实现，以及直接发 HTTP 请求、携带检查项的新实现。每次调用时读取配置开关来选择，切换不需要重启。</li>
  <li><strong>检查项不写死。</strong> 从上游的配置接口读取检查项列表，并按项目缓存。</li>
  <li><strong>利用 422 响应自我修复。</strong> 项目的检查项配置变化后，按旧缓存构造的请求会收到 422，而 422 的响应体中带有当前生效的列表。组件直接用响应体里的列表重建请求重试一次，省去一次额外的读取往返。</li>
  <li><strong>保持异常语义不变。</strong> 两种实现都以客户端库原有的异常类型报告失败，并携带真实的 HTTP 状态码。调用方的重试循环、401 歧义处理和日志都无需改动。</li>
</ul>

<h3 id="39-为什么没有选择工作流引擎">3.9 为什么没有选择工作流引擎</h3>

<p>成熟的工作流引擎能提供持久化执行、自动重试和崩溃恢复，这些正是本方案中手工实现的部分。我们没有采用的原因是：这条流程的步骤数量有限；团队已有的共享基础设施只有关系数据库；而真正困难的部分（外部响应的歧义、回调的过期判断、以远端状态为准的幂等）是业务语义，引擎并不能替我们解决。代价是部分持久化执行能力需要自己补齐，见第 5 节。</p>

<h2 id="4-成果">4. 成果</h2>

<ul>
  <li><strong>状态可查询。</strong> 每个请求都能通过 <code class="language-plaintext highlighter-rouge">operationId</code> 查到所处阶段和停止的具体步骤，排障不需要翻日志重建过程。</li>
  <li><strong>失败可见。</strong> 线程池过载、步骤异常、流水线失败都会落成明确的失败状态，不会表现为永远”处理中”。</li>
  <li><strong>对事件鲁棒。</strong> 重复投递、迟到回调、属于旧申请的回调不会改变当前申请的状态。</li>
  <li><strong>重入安全。</strong> 运维从中断处重新触发步骤时，已合并的请求直接收尾，已批准的账号跳过批准，不会制造新的失败。</li>
  <li><strong>流程可演进。</strong> 流程顺序切换通过配置完成，发布期间的在途审批无需排空；上游接口变更通过开关切换，调用方无需修改，切换无需重启。</li>
  <li><strong>丢失回调可自动收敛。</strong> 定时对账把滞留记录推进到终态，并保证成功指标只计一次。</li>
</ul>

<h2 id="5-已知限制与后续计划">5. 已知限制与后续计划</h2>

<ul>
  <li><strong>合法转移没有集中定义。</strong> 乐观并发校验比较的是”刚读到的状态”，只能防止丢失更新，不能防止非法的 from→to 转移。合法性目前分散在入口守卫、可重新申请集合和推进方法的前置条件中。后续计划引入集中的转移表，所有写入都经过它校验。</li>
  <li><strong>执行状态不持久。</strong> 长流程运行在进程内线程池中，等待通过线程休眠实现。进程在步骤中途崩溃时，记录会停在”进行中”。批量路径已有超时扫描，单请求路径目前依赖运维工具恢复。后续计划为各类”进行中”状态增加超时扫描，或把等待改为由调度器驱动。</li>
  <li><strong>对账依赖推断。</strong> 水位线方法依赖 MR 编号单调和部署流水线的执行语义，且回调表不含业务维度，比较是全局的。如果回调能携带业务标识，可以改为精确匹配。</li>
  <li><strong>流程代码集中在一个大类中。</strong> 这增加了整体审计状态机的成本，后续计划按阶段拆分。</li>
</ul>]]></content><author><name></name></author><category term="zh" /><summary type="html"><![CDATA[一个跨审批系统、GitLab 与 CI 的长流程如何做到状态可观察、失败可见、对重复与迟到事件鲁棒，并在不中断在途请求的前提下演进。]]></summary></entry></feed>