[{"data":1,"prerenderedAt":3163},["ShallowReactive",2],{"page-\u002Fgetting-started-with-python-apis-for-builders\u002Funderstanding-rest-vs-graphql\u002Fwhen-to-use-webhooks-instead-of-polling\u002F":3,"faq-schema-\u002Fgetting-started-with-python-apis-for-builders\u002Funderstanding-rest-vs-graphql\u002Fwhen-to-use-webhooks-instead-of-polling\u002F":3142},{"id":4,"title":5,"body":6,"description":3132,"extension":3133,"meta":3134,"navigation":217,"path":3138,"seo":3139,"stem":3140,"__hash__":3141},"content\u002Fgetting-started-with-python-apis-for-builders\u002Funderstanding-rest-vs-graphql\u002Fwhen-to-use-webhooks-instead-of-polling\u002Findex.md","When to Use Webhooks Instead of Polling",{"type":7,"value":8,"toc":3120},"minimark",[9,13,23,26,31,34,37,40,160,168,176,933,937,940,943,1012,1024,1028,1031,1149,1156,1543,1564,1568,1571,1574,1660,1672,2325,2332,2336,2339,2461,2472,2925,2938,2942,3020,3023,3027,3035,3039,3042,3046,3052,3058,3064,3070,3076,3080,3116],[10,11,5],"h1",{"id":12},"when-to-use-webhooks-instead-of-polling",[14,15,16,17,22],"p",{},"You have a third-party integration that needs to react to something changing somewhere else: an order was paid, a document finished processing, a lead replied. You can ask repeatedly, or you can be told. That single decision sets your compute bill, your p50 freshness, and how often you get rate limited for the next two years. Part of the ",[18,19,21],"a",{"href":20},"\u002Fgetting-started-with-python-apis-for-builders\u002Funderstanding-rest-vs-graphql\u002F","Understanding REST vs GraphQL"," guide, because the protocol conversation is incomplete until you decide who initiates the call.",[14,24,25],{},"The short version: webhooks win on almost every commercial axis the moment your event rate per watched resource drops below your freshness requirement, which is nearly always. But a webhook-only design is fragile, and a polling-only design is expensive. The answer you ship is a hybrid, and this page gives you the arithmetic and the code for it.",[27,28,30],"h2",{"id":29},"the-cost-crossover-is-arithmetic-not-opinion","The Cost Crossover Is Arithmetic, Not Opinion",[14,32,33],{},"Polling cost scales with the number of watched resources multiplied by the polling frequency. Webhook cost scales with the number of actual events. Those are completely different curves, and they cross at a point you can calculate before writing a line of code.",[14,35,36],{},"Take a realistic side-hustle scale: 500 connected customer accounts, each of which you check for new activity. Poll every 60 seconds and you issue 500 × 60 × 24 × 30 = 21.6 million outbound requests a month. At roughly 120 ms of process time per request — connection reuse, TLS resumption, JSON decode, a database comparison — that is 720 CPU-hours. A calendar month is 730 hours. You are burning one saturated vCPU continuously, forever, to ask a question whose answer is \"nothing changed\" more than 99% of the time.",[14,38,39],{},"Now count the webhook side. If each of those 500 accounts genuinely produces 40 meaningful events a month, that is 20,000 inbound requests. Same 120 ms of work each, and the whole month costs 40 CPU-minutes. The difference is not a percentage. It is three orders of magnitude.",[41,42,50,51,50,55,50,59,50,66,50,74,50,82,50,90,50,96,50,101,50,106,50,111,50,114,50,120,50,125,50,128,50,133,50,138,50,141,50,147,50,152,50,155],"svg",{"viewBox":43,"role":44,"ariaLabelledBy":45,"xmlns":48,"style":49},"0 0 720 300","img",[46,47],"wvp-cost-t","wvp-cost-d","http:\u002F\u002Fwww.w3.org\u002F2000\u002Fsvg","width:100%;height:auto;margin:1.5rem 0;font-family:var(--font-sans);","\n  ",[52,53,54],"title",{"id":46},"Monthly request volume by polling interval versus webhooks",[56,57,58],"desc",{"id":47},"Log-scaled bars for 500 watched accounts: 21.6 million requests a month at a 60 second interval, 4.32 million at 5 minutes, 1.44 million at 15 minutes, 360 thousand hourly, and 20 thousand with webhooks.",[60,61],"rect",{"x":62,"y":62,"width":63,"height":64,"fill":65},"0","720","300","var(--c-surface)",[67,68,73],"text",{"x":69,"y":70,"fill":71,"style":72},"360","26","var(--c-text)","text-anchor:middle;font-size:14;font-family:var(--font-sans);","Requests per month, 500 watched accounts (log scale)",[75,76],"line",{"x1":77,"y1":78,"x2":79,"y2":78,"stroke":80,"style":81},"40","240","700","var(--c-border)","stroke-width:2;",[60,83],{"x":84,"y":85,"width":85,"height":86,"rx":87,"fill":88,"stroke":89,"style":81},"50","80","160","4","var(--c-surface-alt)","var(--c-coral)",[67,91,95],{"x":92,"y":93,"fill":71,"style":94},"90","72","text-anchor:middle;font-size:12;font-family:var(--font-sans);","21.6M",[67,97,100],{"x":92,"y":98,"fill":99,"style":94},"262","var(--c-text-muted)","60 s poll",[60,102],{"x":103,"y":104,"width":85,"height":105,"rx":87,"fill":88,"stroke":89,"style":81},"180","114","126",[67,107,110],{"x":108,"y":109,"fill":71,"style":94},"220","106","4.32M",[67,112,113],{"x":108,"y":98,"fill":99,"style":94},"5 min",[60,115],{"x":116,"y":117,"width":85,"height":118,"rx":87,"fill":88,"stroke":119,"style":81},"310","136","104","var(--c-yellow)",[67,121,124],{"x":122,"y":123,"fill":71,"style":94},"350","128","1.44M",[67,126,127],{"x":122,"y":98,"fill":99,"style":94},"15 min",[60,129],{"x":130,"y":131,"width":85,"height":132,"rx":87,"fill":88,"stroke":119,"style":81},"440","165","75",[67,134,137],{"x":135,"y":136,"fill":71,"style":94},"480","157","360k",[67,139,140],{"x":135,"y":98,"fill":99,"style":94},"60 min",[60,142],{"x":143,"y":144,"width":85,"height":145,"rx":87,"fill":88,"stroke":146,"style":81},"570","226","14","var(--c-teal)",[67,148,151],{"x":149,"y":150,"fill":71,"style":94},"610","218","20k",[67,153,154],{"x":149,"y":98,"fill":99,"style":94},"webhooks",[67,156,159],{"x":69,"y":157,"fill":99,"style":158},"286","text-anchor:middle;font-size:11;font-family:var(--font-sans);","webhook volume assumes 40 real events per account per month",[14,161,162,163,167],{},"The crossover rule falls out of that comparison in one line: ",[164,165,166],"strong",{},"polling is cheaper only when each watched resource changes more often than once per polling interval."," If a customer's Stripe account fires an event every ten seconds and you need one-minute freshness, polling batches six events into one request and wins. That situation is rare. For almost every integration a builder ships — payments, CRM records, file processing, inbox sync — resources sit idle for hours and change in bursts.",[14,169,170,171,175],{},"Encode the model rather than guessing. This script gives you a defensible number for a pricing conversation, and it pairs with the deeper treatment in ",[18,172,174],{"href":173},"\u002Fbuilding-monetizing-api-driven-micro-saas\u002Fdesigning-api-pricing-tiers\u002Fcalculating-cost-per-api-request\u002F","calculating cost per API request",".",[177,178,183],"pre",{"className":179,"code":180,"language":181,"meta":182,"style":182},"language-python shiki shiki-themes github-light github-dark","import os\nfrom dataclasses import dataclass\n\nSECONDS_PER_MONTH = 30 * 24 * 3600\n\n\n@dataclass(frozen=True)\nclass Workload:\n    accounts: int\n    poll_interval_s: int\n    events_per_account_month: int\n    cpu_seconds_per_request: float\n    cpu_hour_cost: float\n\n    @property\n    def poll_requests(self) -> int:\n        return self.accounts * (SECONDS_PER_MONTH \u002F\u002F self.poll_interval_s)\n\n    @property\n    def webhook_requests(self) -> int:\n        return self.accounts * self.events_per_account_month\n\n    def cost(self, requests: int) -> float:\n        return requests * self.cpu_seconds_per_request \u002F 3600 * self.cpu_hour_cost\n\n\ndef load_workload() -> Workload:\n    return Workload(\n        accounts=int(os.getenv(\"WATCHED_ACCOUNTS\", \"500\")),\n        poll_interval_s=int(os.getenv(\"POLL_INTERVAL_SECONDS\", \"60\")),\n        events_per_account_month=int(os.getenv(\"EVENTS_PER_ACCOUNT\", \"40\")),\n        cpu_seconds_per_request=float(os.getenv(\"CPU_SECONDS_PER_REQUEST\", \"0.12\")),\n        cpu_hour_cost=float(os.getenv(\"CPU_HOUR_COST\", \"0.034\")),\n    )\n\n\nif __name__ == \"__main__\":\n    w = load_workload()\n    poll_cost = w.cost(w.poll_requests)\n    hook_cost = w.cost(w.webhook_requests)\n    ratio = poll_cost \u002F max(hook_cost, 1e-9)\n\n    match ratio:\n        case r if r > 20:\n            verdict = \"webhooks, decisively\"\n        case r if r > 2:\n            verdict = \"webhooks, with a reconciliation sweep\"\n        case _:\n            verdict = \"polling is competitive, keep it simple\"\n\n    print(f\"poll:    {w.poll_requests:>12,} req  ${poll_cost:,.2f}\")\n    print(f\"webhook: {w.webhook_requests:>12,} req  ${hook_cost:,.2f}\")\n    print(f\"ratio {ratio:,.1f}x -> {verdict}\")\n","python","",[184,185,186,198,212,219,243,248,253,276,288,297,305,313,322,330,335,344,361,389,394,401,415,431,436,457,485,490,495,507,516,542,564,586,608,630,636,641,646,663,674,685,696,720,725,734,755,766,784,794,802,812,817,861,897],"code",{"__ignoreMap":182},[187,188,190,194],"span",{"class":75,"line":189},1,[187,191,193],{"class":192},"szBVR","import",[187,195,197],{"class":196},"sVt8B"," os\n",[187,199,201,204,207,209],{"class":75,"line":200},2,[187,202,203],{"class":192},"from",[187,205,206],{"class":196}," dataclasses ",[187,208,193],{"class":192},[187,210,211],{"class":196}," dataclass\n",[187,213,215],{"class":75,"line":214},3,[187,216,218],{"emptyLinePlaceholder":217},true,"\n",[187,220,222,226,229,232,235,238,240],{"class":75,"line":221},4,[187,223,225],{"class":224},"sj4cs","SECONDS_PER_MONTH",[187,227,228],{"class":192}," =",[187,230,231],{"class":224}," 30",[187,233,234],{"class":192}," *",[187,236,237],{"class":224}," 24",[187,239,234],{"class":192},[187,241,242],{"class":224}," 3600\n",[187,244,246],{"class":75,"line":245},5,[187,247,218],{"emptyLinePlaceholder":217},[187,249,251],{"class":75,"line":250},6,[187,252,218],{"emptyLinePlaceholder":217},[187,254,256,260,263,267,270,273],{"class":75,"line":255},7,[187,257,259],{"class":258},"sScJk","@dataclass",[187,261,262],{"class":196},"(",[187,264,266],{"class":265},"s4XuR","frozen",[187,268,269],{"class":192},"=",[187,271,272],{"class":224},"True",[187,274,275],{"class":196},")\n",[187,277,279,282,285],{"class":75,"line":278},8,[187,280,281],{"class":192},"class",[187,283,284],{"class":258}," Workload",[187,286,287],{"class":196},":\n",[187,289,291,294],{"class":75,"line":290},9,[187,292,293],{"class":196},"    accounts: ",[187,295,296],{"class":224},"int\n",[187,298,300,303],{"class":75,"line":299},10,[187,301,302],{"class":196},"    poll_interval_s: ",[187,304,296],{"class":224},[187,306,308,311],{"class":75,"line":307},11,[187,309,310],{"class":196},"    events_per_account_month: ",[187,312,296],{"class":224},[187,314,316,319],{"class":75,"line":315},12,[187,317,318],{"class":196},"    cpu_seconds_per_request: ",[187,320,321],{"class":224},"float\n",[187,323,325,328],{"class":75,"line":324},13,[187,326,327],{"class":196},"    cpu_hour_cost: ",[187,329,321],{"class":224},[187,331,333],{"class":75,"line":332},14,[187,334,218],{"emptyLinePlaceholder":217},[187,336,338,341],{"class":75,"line":337},15,[187,339,340],{"class":258},"    @",[187,342,343],{"class":224},"property\n",[187,345,347,350,353,356,359],{"class":75,"line":346},16,[187,348,349],{"class":192},"    def",[187,351,352],{"class":258}," poll_requests",[187,354,355],{"class":196},"(self) -> ",[187,357,358],{"class":224},"int",[187,360,287],{"class":196},[187,362,364,367,370,373,376,379,381,384,386],{"class":75,"line":363},17,[187,365,366],{"class":192},"        return",[187,368,369],{"class":224}," self",[187,371,372],{"class":196},".accounts ",[187,374,375],{"class":192},"*",[187,377,378],{"class":196}," (",[187,380,225],{"class":224},[187,382,383],{"class":192}," \u002F\u002F",[187,385,369],{"class":224},[187,387,388],{"class":196},".poll_interval_s)\n",[187,390,392],{"class":75,"line":391},18,[187,393,218],{"emptyLinePlaceholder":217},[187,395,397,399],{"class":75,"line":396},19,[187,398,340],{"class":258},[187,400,343],{"class":224},[187,402,404,406,409,411,413],{"class":75,"line":403},20,[187,405,349],{"class":192},[187,407,408],{"class":258}," webhook_requests",[187,410,355],{"class":196},[187,412,358],{"class":224},[187,414,287],{"class":196},[187,416,418,420,422,424,426,428],{"class":75,"line":417},21,[187,419,366],{"class":192},[187,421,369],{"class":224},[187,423,372],{"class":196},[187,425,375],{"class":192},[187,427,369],{"class":224},[187,429,430],{"class":196},".events_per_account_month\n",[187,432,434],{"class":75,"line":433},22,[187,435,218],{"emptyLinePlaceholder":217},[187,437,439,441,444,447,449,452,455],{"class":75,"line":438},23,[187,440,349],{"class":192},[187,442,443],{"class":258}," cost",[187,445,446],{"class":196},"(self, requests: ",[187,448,358],{"class":224},[187,450,451],{"class":196},") -> ",[187,453,454],{"class":224},"float",[187,456,287],{"class":196},[187,458,460,462,465,467,469,472,475,478,480,482],{"class":75,"line":459},24,[187,461,366],{"class":192},[187,463,464],{"class":196}," requests ",[187,466,375],{"class":192},[187,468,369],{"class":224},[187,470,471],{"class":196},".cpu_seconds_per_request ",[187,473,474],{"class":192},"\u002F",[187,476,477],{"class":224}," 3600",[187,479,234],{"class":192},[187,481,369],{"class":224},[187,483,484],{"class":196},".cpu_hour_cost\n",[187,486,488],{"class":75,"line":487},25,[187,489,218],{"emptyLinePlaceholder":217},[187,491,493],{"class":75,"line":492},26,[187,494,218],{"emptyLinePlaceholder":217},[187,496,498,501,504],{"class":75,"line":497},27,[187,499,500],{"class":192},"def",[187,502,503],{"class":258}," load_workload",[187,505,506],{"class":196},"() -> Workload:\n",[187,508,510,513],{"class":75,"line":509},28,[187,511,512],{"class":192},"    return",[187,514,515],{"class":196}," Workload(\n",[187,517,519,522,524,526,529,533,536,539],{"class":75,"line":518},29,[187,520,521],{"class":265},"        accounts",[187,523,269],{"class":192},[187,525,358],{"class":224},[187,527,528],{"class":196},"(os.getenv(",[187,530,532],{"class":531},"sZZnC","\"WATCHED_ACCOUNTS\"",[187,534,535],{"class":196},", ",[187,537,538],{"class":531},"\"500\"",[187,540,541],{"class":196},")),\n",[187,543,545,548,550,552,554,557,559,562],{"class":75,"line":544},30,[187,546,547],{"class":265},"        poll_interval_s",[187,549,269],{"class":192},[187,551,358],{"class":224},[187,553,528],{"class":196},[187,555,556],{"class":531},"\"POLL_INTERVAL_SECONDS\"",[187,558,535],{"class":196},[187,560,561],{"class":531},"\"60\"",[187,563,541],{"class":196},[187,565,567,570,572,574,576,579,581,584],{"class":75,"line":566},31,[187,568,569],{"class":265},"        events_per_account_month",[187,571,269],{"class":192},[187,573,358],{"class":224},[187,575,528],{"class":196},[187,577,578],{"class":531},"\"EVENTS_PER_ACCOUNT\"",[187,580,535],{"class":196},[187,582,583],{"class":531},"\"40\"",[187,585,541],{"class":196},[187,587,589,592,594,596,598,601,603,606],{"class":75,"line":588},32,[187,590,591],{"class":265},"        cpu_seconds_per_request",[187,593,269],{"class":192},[187,595,454],{"class":224},[187,597,528],{"class":196},[187,599,600],{"class":531},"\"CPU_SECONDS_PER_REQUEST\"",[187,602,535],{"class":196},[187,604,605],{"class":531},"\"0.12\"",[187,607,541],{"class":196},[187,609,611,614,616,618,620,623,625,628],{"class":75,"line":610},33,[187,612,613],{"class":265},"        cpu_hour_cost",[187,615,269],{"class":192},[187,617,454],{"class":224},[187,619,528],{"class":196},[187,621,622],{"class":531},"\"CPU_HOUR_COST\"",[187,624,535],{"class":196},[187,626,627],{"class":531},"\"0.034\"",[187,629,541],{"class":196},[187,631,633],{"class":75,"line":632},34,[187,634,635],{"class":196},"    )\n",[187,637,639],{"class":75,"line":638},35,[187,640,218],{"emptyLinePlaceholder":217},[187,642,644],{"class":75,"line":643},36,[187,645,218],{"emptyLinePlaceholder":217},[187,647,649,652,655,658,661],{"class":75,"line":648},37,[187,650,651],{"class":192},"if",[187,653,654],{"class":224}," __name__",[187,656,657],{"class":192}," ==",[187,659,660],{"class":531}," \"__main__\"",[187,662,287],{"class":196},[187,664,666,669,671],{"class":75,"line":665},38,[187,667,668],{"class":196},"    w ",[187,670,269],{"class":192},[187,672,673],{"class":196}," load_workload()\n",[187,675,677,680,682],{"class":75,"line":676},39,[187,678,679],{"class":196},"    poll_cost ",[187,681,269],{"class":192},[187,683,684],{"class":196}," w.cost(w.poll_requests)\n",[187,686,688,691,693],{"class":75,"line":687},40,[187,689,690],{"class":196},"    hook_cost ",[187,692,269],{"class":192},[187,694,695],{"class":196}," w.cost(w.webhook_requests)\n",[187,697,699,702,704,707,709,712,715,718],{"class":75,"line":698},41,[187,700,701],{"class":196},"    ratio ",[187,703,269],{"class":192},[187,705,706],{"class":196}," poll_cost ",[187,708,474],{"class":192},[187,710,711],{"class":224}," max",[187,713,714],{"class":196},"(hook_cost, ",[187,716,717],{"class":224},"1e-9",[187,719,275],{"class":196},[187,721,723],{"class":75,"line":722},42,[187,724,218],{"emptyLinePlaceholder":217},[187,726,728,731],{"class":75,"line":727},43,[187,729,730],{"class":192},"    match",[187,732,733],{"class":196}," ratio:\n",[187,735,737,740,743,745,747,750,753],{"class":75,"line":736},44,[187,738,739],{"class":192},"        case",[187,741,742],{"class":196}," r ",[187,744,651],{"class":192},[187,746,742],{"class":196},[187,748,749],{"class":192},">",[187,751,752],{"class":224}," 20",[187,754,287],{"class":196},[187,756,758,761,763],{"class":75,"line":757},45,[187,759,760],{"class":196},"            verdict ",[187,762,269],{"class":192},[187,764,765],{"class":531}," \"webhooks, decisively\"\n",[187,767,769,771,773,775,777,779,782],{"class":75,"line":768},46,[187,770,739],{"class":192},[187,772,742],{"class":196},[187,774,651],{"class":192},[187,776,742],{"class":196},[187,778,749],{"class":192},[187,780,781],{"class":224}," 2",[187,783,287],{"class":196},[187,785,787,789,791],{"class":75,"line":786},47,[187,788,760],{"class":196},[187,790,269],{"class":192},[187,792,793],{"class":531}," \"webhooks, with a reconciliation sweep\"\n",[187,795,797,799],{"class":75,"line":796},48,[187,798,739],{"class":192},[187,800,801],{"class":196}," _:\n",[187,803,805,807,809],{"class":75,"line":804},49,[187,806,760],{"class":196},[187,808,269],{"class":192},[187,810,811],{"class":531}," \"polling is competitive, keep it simple\"\n",[187,813,815],{"class":75,"line":814},50,[187,816,218],{"emptyLinePlaceholder":217},[187,818,820,823,825,828,831,834,837,840,843,846,848,851,854,856,859],{"class":75,"line":819},51,[187,821,822],{"class":224},"    print",[187,824,262],{"class":196},[187,826,827],{"class":192},"f",[187,829,830],{"class":531},"\"poll:    ",[187,832,833],{"class":224},"{",[187,835,836],{"class":196},"w.poll_requests",[187,838,839],{"class":192},":>12,",[187,841,842],{"class":224},"}",[187,844,845],{"class":531}," req  $",[187,847,833],{"class":224},[187,849,850],{"class":196},"poll_cost",[187,852,853],{"class":192},":,.2f",[187,855,842],{"class":224},[187,857,858],{"class":531},"\"",[187,860,275],{"class":196},[187,862,864,866,868,870,873,875,878,880,882,884,886,889,891,893,895],{"class":75,"line":863},52,[187,865,822],{"class":224},[187,867,262],{"class":196},[187,869,827],{"class":192},[187,871,872],{"class":531},"\"webhook: ",[187,874,833],{"class":224},[187,876,877],{"class":196},"w.webhook_requests",[187,879,839],{"class":192},[187,881,842],{"class":224},[187,883,845],{"class":531},[187,885,833],{"class":224},[187,887,888],{"class":196},"hook_cost",[187,890,853],{"class":192},[187,892,842],{"class":224},[187,894,858],{"class":531},[187,896,275],{"class":196},[187,898,900,902,904,906,909,911,914,917,919,922,924,927,929,931],{"class":75,"line":899},53,[187,901,822],{"class":224},[187,903,262],{"class":196},[187,905,827],{"class":192},[187,907,908],{"class":531},"\"ratio ",[187,910,833],{"class":224},[187,912,913],{"class":196},"ratio",[187,915,916],{"class":192},":,.1f",[187,918,842],{"class":224},[187,920,921],{"class":531},"x -> ",[187,923,833],{"class":224},[187,925,926],{"class":196},"verdict",[187,928,842],{"class":224},[187,930,858],{"class":531},[187,932,275],{"class":196},[27,934,936],{"id":935},"latency-is-the-axis-that-actually-sells","Latency Is the Axis That Actually Sells",[14,938,939],{},"Cost convinces you. Latency convinces your customers. Polling produces a sawtooth staleness curve: immediately after a poll your data is fresh, and just before the next one it is a full interval old. Mean detection latency is half the interval, worst case is the whole interval plus the provider's own write propagation delay. A five-minute poll means the average customer waits 150 seconds to see their payment confirmed, and the unlucky ones wait 300.",[14,941,942],{},"A webhook flattens that curve. Providers typically dispatch within one to three seconds of the state change, so your p50 detection latency drops to single-digit seconds and your p99 is governed by their retry queue rather than your cron schedule. That is the difference between a dashboard that feels live and one that feels broken.",[41,944,50,949,50,952,50,955,50,958,50,962,50,967,50,969,50,975,50,979,50,984,50,989,50,994,50,999,50,1005,50,1008],{"viewBox":945,"role":44,"ariaLabelledBy":946,"xmlns":48,"style":49},"0 0 720 250",[947,948],"wvp-stale-t","wvp-stale-d",[52,950,951],{"id":947},"Data staleness over time for five-minute polling versus webhook delivery",[56,953,954],{"id":948},"Polling produces a sawtooth that climbs to 300 seconds of staleness before each poll resets it, while webhook delivery holds a nearly flat line close to two seconds.",[60,956],{"x":62,"y":62,"width":63,"height":957,"fill":65},"250",[67,959,961],{"x":69,"y":960,"fill":71,"style":72},"24","How stale is your copy of the data?",[75,963],{"x1":964,"y1":965,"x2":966,"y2":965,"stroke":80,"style":81},"70","200","690",[75,968],{"x1":964,"y1":84,"x2":964,"y2":965,"stroke":80,"style":81},[67,970,974],{"x":971,"y":972,"fill":99,"style":973},"62","64","text-anchor:end;font-size:11;font-family:var(--font-sans);","300 s",[67,976,978],{"x":971,"y":977,"fill":99,"style":973},"204","0 s",[980,981],"path",{"d":982,"fill":983,"stroke":89,"style":81},"M70 200 L192 60 L192 200 L314 60 L314 200 L436 60 L436 200 L558 60 L558 200 L680 60","none",[75,985],{"x1":964,"y1":986,"x2":987,"y2":986,"stroke":146,"style":988},"194","680","stroke-width:3;",[67,990,993],{"x":991,"y":992,"fill":99,"style":158},"380","228","25 minutes of wall clock time",[60,995],{"x":130,"y":996,"width":145,"height":997,"rx":998,"fill":88,"stroke":89,"style":81},"76","10","2",[67,1000,1004],{"x":1001,"y":1002,"fill":71,"style":1003},"462","86","text-anchor:start;font-size:12;font-family:var(--font-sans);","5 min poll",[60,1006],{"x":130,"y":1007,"width":145,"height":997,"rx":998,"fill":88,"stroke":146,"style":81},"100",[67,1009,1011],{"x":1001,"y":1010,"fill":71,"style":1003},"110","webhook",[14,1013,1014,1015,1019,1020,175],{},"There is a second, meaner latency effect. Most providers cap you at something like 100 requests per minute per token. Five hundred accounts polled every minute needs 500 requests per minute, so you either shard across tokens or you stagger — and staggering silently stretches your real freshness from one minute to five while your monitoring still says \"polling every 60 s\". If you have ever chased that discrepancy, the mechanics are laid out in ",[18,1016,1018],{"href":1017},"\u002Fgetting-started-with-python-apis-for-builders\u002Fmaking-http-requests-with-requests-library\u002Fdebugging-429-too-many-requests-errors\u002F","debugging 429 Too Many Requests errors"," and the budgeting side in ",[18,1021,1023],{"href":1022},"\u002Fgetting-started-with-python-apis-for-builders\u002Fmaking-http-requests-with-requests-library\u002Fbest-practices-for-api-rate-limiting\u002F","best practices for API rate limiting",[27,1025,1027],{"id":1026},"ship-the-receiver-as-a-thin-fast-acknowledgement","Ship the Receiver as a Thin, Fast Acknowledgement",[14,1029,1030],{},"A webhook endpoint is a public, unauthenticated-by-default door into your system, and the provider will disable it if you are slow. Every serious provider expects a 2xx within a few seconds and retries with exponential backoff otherwise. So the receiver does exactly three things: verify the signature, persist the raw event, return 202. All real work happens elsewhere.",[41,1032,50,1037,50,1040,50,1043,50,1046,50,1053,50,1056,50,1060,50,1063,50,1066,50,1070,50,1073,50,1077,50,1081,50,1083,50,1085,50,1087,50,1090,50,1093,50,1097,50,1102,50,1107,50,1111,50,1114,50,1119,50,1123,50,1126,50,1130,50,1133,50,1136,50,1141,50,1145],{"viewBox":1033,"role":44,"ariaLabelledBy":1034,"xmlns":48,"style":49},"0 0 720 320",[1035,1036],"wvp-seq-t","wvp-seq-d",[52,1038,1039],{"id":1035},"Webhook receive sequence from provider to background worker",[56,1041,1042],{"id":1036},"The provider posts a signed event, the API verifies the HMAC, enqueues a job, returns 202 Accepted in under fifty milliseconds, and a background worker performs the idempotent write.",[60,1044],{"x":62,"y":62,"width":63,"height":1045,"fill":65},"320",[60,1047],{"x":1048,"y":70,"width":1049,"height":1050,"rx":1051,"fill":88,"stroke":1052,"style":81},"25","130","38","8","var(--c-blue)",[67,1054,1055],{"x":92,"y":84,"fill":71,"style":94},"Provider",[60,1057],{"x":1058,"y":70,"width":1059,"height":1050,"rx":1051,"fill":88,"stroke":146,"style":81},"225","150",[67,1061,1062],{"x":64,"y":84,"fill":71,"style":94},"FastAPI route",[60,1064],{"x":1065,"y":70,"width":1010,"height":1050,"rx":1051,"fill":88,"stroke":119,"style":81},"435",[67,1067,1069],{"x":1068,"y":84,"fill":71,"style":94},"490","Queue",[60,1071],{"x":1072,"y":70,"width":1010,"height":1050,"rx":1051,"fill":88,"stroke":89,"style":81},"585",[67,1074,1076],{"x":1075,"y":84,"fill":71,"style":94},"640","Worker",[75,1078],{"x1":92,"y1":972,"x2":92,"y2":1079,"stroke":80,"style":1080},"296","stroke-width:1;",[75,1082],{"x1":64,"y1":972,"x2":64,"y2":1079,"stroke":80,"style":1080},[75,1084],{"x1":1068,"y1":972,"x2":1068,"y2":1079,"stroke":80,"style":1080},[75,1086],{"x1":1075,"y1":972,"x2":1075,"y2":1079,"stroke":80,"style":1080},[75,1088],{"x1":92,"y1":109,"x2":1089,"y2":109,"stroke":1052,"style":81},"298",[980,1091],{"d":1092,"fill":1052},"M298 106 L288 101 L288 111 z",[67,1094,1096],{"x":986,"y":1095,"fill":71,"style":94},"98","POST signed event",[60,1098],{"x":1099,"y":105,"width":1100,"height":70,"rx":1101,"fill":88,"stroke":146,"style":81},"302","120","6",[67,1103,1106],{"x":1104,"y":1105,"fill":71,"style":158},"362","144","verify HMAC",[75,1108],{"x1":64,"y1":1109,"x2":1110,"y2":1109,"stroke":119,"style":81},"176","488",[980,1112],{"d":1113,"fill":119},"M488 176 L478 171 L478 181 z",[67,1115,1118],{"x":1116,"y":1117,"fill":71,"style":94},"394","168","enqueue job",[75,1120],{"x1":1089,"y1":1121,"x2":1122,"y2":1121,"stroke":146,"style":81},"212","92",[980,1124],{"d":1125,"fill":146},"M92 212 L102 207 L102 217 z",[67,1127,1129],{"x":1128,"y":977,"fill":71,"style":94},"196","202 in under 50 ms",[75,1131],{"x1":1068,"y1":957,"x2":1132,"y2":957,"stroke":89,"style":81},"638",[980,1134],{"d":1135,"fill":89},"M638 250 L628 245 L628 255 z",[67,1137,1140],{"x":1138,"y":1139,"fill":71,"style":94},"564","242","dequeue",[60,1142],{"x":1143,"y":1144,"width":86,"height":70,"rx":1101,"fill":88,"stroke":89,"style":81},"530","266",[67,1146,1148],{"x":149,"y":1147,"fill":71,"style":158},"284","idempotent upsert",[14,1150,1151,1152,175],{},"Read the body as raw bytes before anything else. Signature schemes hash the exact transmitted payload, and if you let a framework parse and re-serialise the JSON first, key ordering and whitespace changes will break verification in ways that look like a secret mismatch. This is a natural fit for an async ",[18,1153,1155],{"href":1154},"\u002Fgetting-started-with-python-apis-for-builders\u002Fsetting-up-fastapi\u002F","FastAPI setup",[177,1157,1159],{"className":179,"code":1158,"language":181,"meta":182,"style":182},"import hashlib\nimport hmac\nimport os\n\nfrom fastapi import APIRouter, Header, HTTPException, Request, Response\n\nrouter = APIRouter()\nWEBHOOK_SECRET = os.environ[\"WEBHOOK_SIGNING_SECRET\"].encode()\nMAX_BODY_BYTES = int(os.getenv(\"WEBHOOK_MAX_BODY_BYTES\", \"1048576\"))\n\n\ndef valid_signature(payload: bytes, provided: str) -> bool:\n    expected = hmac.new(WEBHOOK_SECRET, payload, hashlib.sha256).hexdigest()\n    return hmac.compare_digest(expected, provided)\n\n\n@router.post(\"\u002Fwebhooks\u002Fprovider\", status_code=202)\nasync def receive(request: Request, x_signature: str = Header(default=\"\")) -> Response:\n    payload = await request.body()\n    if len(payload) > MAX_BODY_BYTES:\n        raise HTTPException(status_code=413, detail=\"payload too large\")\n    if not valid_signature(payload, x_signature):\n        raise HTTPException(status_code=401, detail=\"bad signature\")\n\n    event = await request.json()\n    await store_raw_event(\n        event_id=event[\"id\"],\n        event_type=event[\"type\"],\n        payload=payload,\n    )\n    return Response(status_code=202)\n",[184,1160,1161,1168,1175,1181,1185,1197,1201,1211,1227,1250,1254,1258,1284,1299,1306,1310,1314,1336,1368,1381,1399,1426,1436,1460,1464,1476,1484,1500,1514,1524,1528],{"__ignoreMap":182},[187,1162,1163,1165],{"class":75,"line":189},[187,1164,193],{"class":192},[187,1166,1167],{"class":196}," hashlib\n",[187,1169,1170,1172],{"class":75,"line":200},[187,1171,193],{"class":192},[187,1173,1174],{"class":196}," hmac\n",[187,1176,1177,1179],{"class":75,"line":214},[187,1178,193],{"class":192},[187,1180,197],{"class":196},[187,1182,1183],{"class":75,"line":221},[187,1184,218],{"emptyLinePlaceholder":217},[187,1186,1187,1189,1192,1194],{"class":75,"line":245},[187,1188,203],{"class":192},[187,1190,1191],{"class":196}," fastapi ",[187,1193,193],{"class":192},[187,1195,1196],{"class":196}," APIRouter, Header, HTTPException, Request, Response\n",[187,1198,1199],{"class":75,"line":250},[187,1200,218],{"emptyLinePlaceholder":217},[187,1202,1203,1206,1208],{"class":75,"line":255},[187,1204,1205],{"class":196},"router ",[187,1207,269],{"class":192},[187,1209,1210],{"class":196}," APIRouter()\n",[187,1212,1213,1216,1218,1221,1224],{"class":75,"line":278},[187,1214,1215],{"class":224},"WEBHOOK_SECRET",[187,1217,228],{"class":192},[187,1219,1220],{"class":196}," os.environ[",[187,1222,1223],{"class":531},"\"WEBHOOK_SIGNING_SECRET\"",[187,1225,1226],{"class":196},"].encode()\n",[187,1228,1229,1232,1234,1237,1239,1242,1244,1247],{"class":75,"line":290},[187,1230,1231],{"class":224},"MAX_BODY_BYTES",[187,1233,228],{"class":192},[187,1235,1236],{"class":224}," int",[187,1238,528],{"class":196},[187,1240,1241],{"class":531},"\"WEBHOOK_MAX_BODY_BYTES\"",[187,1243,535],{"class":196},[187,1245,1246],{"class":531},"\"1048576\"",[187,1248,1249],{"class":196},"))\n",[187,1251,1252],{"class":75,"line":299},[187,1253,218],{"emptyLinePlaceholder":217},[187,1255,1256],{"class":75,"line":307},[187,1257,218],{"emptyLinePlaceholder":217},[187,1259,1260,1262,1265,1268,1271,1274,1277,1279,1282],{"class":75,"line":315},[187,1261,500],{"class":192},[187,1263,1264],{"class":258}," valid_signature",[187,1266,1267],{"class":196},"(payload: ",[187,1269,1270],{"class":224},"bytes",[187,1272,1273],{"class":196},", provided: ",[187,1275,1276],{"class":224},"str",[187,1278,451],{"class":196},[187,1280,1281],{"class":224},"bool",[187,1283,287],{"class":196},[187,1285,1286,1289,1291,1294,1296],{"class":75,"line":324},[187,1287,1288],{"class":196},"    expected ",[187,1290,269],{"class":192},[187,1292,1293],{"class":196}," hmac.new(",[187,1295,1215],{"class":224},[187,1297,1298],{"class":196},", payload, hashlib.sha256).hexdigest()\n",[187,1300,1301,1303],{"class":75,"line":332},[187,1302,512],{"class":192},[187,1304,1305],{"class":196}," hmac.compare_digest(expected, provided)\n",[187,1307,1308],{"class":75,"line":337},[187,1309,218],{"emptyLinePlaceholder":217},[187,1311,1312],{"class":75,"line":346},[187,1313,218],{"emptyLinePlaceholder":217},[187,1315,1316,1319,1321,1324,1326,1329,1331,1334],{"class":75,"line":363},[187,1317,1318],{"class":258},"@router.post",[187,1320,262],{"class":196},[187,1322,1323],{"class":531},"\"\u002Fwebhooks\u002Fprovider\"",[187,1325,535],{"class":196},[187,1327,1328],{"class":265},"status_code",[187,1330,269],{"class":192},[187,1332,1333],{"class":224},"202",[187,1335,275],{"class":196},[187,1337,1338,1341,1344,1347,1350,1352,1354,1357,1360,1362,1365],{"class":75,"line":391},[187,1339,1340],{"class":192},"async",[187,1342,1343],{"class":192}," def",[187,1345,1346],{"class":258}," receive",[187,1348,1349],{"class":196},"(request: Request, x_signature: ",[187,1351,1276],{"class":224},[187,1353,228],{"class":192},[187,1355,1356],{"class":196}," Header(",[187,1358,1359],{"class":265},"default",[187,1361,269],{"class":192},[187,1363,1364],{"class":531},"\"\"",[187,1366,1367],{"class":196},")) -> Response:\n",[187,1369,1370,1373,1375,1378],{"class":75,"line":396},[187,1371,1372],{"class":196},"    payload ",[187,1374,269],{"class":192},[187,1376,1377],{"class":192}," await",[187,1379,1380],{"class":196}," request.body()\n",[187,1382,1383,1386,1389,1392,1394,1397],{"class":75,"line":403},[187,1384,1385],{"class":192},"    if",[187,1387,1388],{"class":224}," len",[187,1390,1391],{"class":196},"(payload) ",[187,1393,749],{"class":192},[187,1395,1396],{"class":224}," MAX_BODY_BYTES",[187,1398,287],{"class":196},[187,1400,1401,1404,1407,1409,1411,1414,1416,1419,1421,1424],{"class":75,"line":417},[187,1402,1403],{"class":192},"        raise",[187,1405,1406],{"class":196}," HTTPException(",[187,1408,1328],{"class":265},[187,1410,269],{"class":192},[187,1412,1413],{"class":224},"413",[187,1415,535],{"class":196},[187,1417,1418],{"class":265},"detail",[187,1420,269],{"class":192},[187,1422,1423],{"class":531},"\"payload too large\"",[187,1425,275],{"class":196},[187,1427,1428,1430,1433],{"class":75,"line":433},[187,1429,1385],{"class":192},[187,1431,1432],{"class":192}," not",[187,1434,1435],{"class":196}," valid_signature(payload, x_signature):\n",[187,1437,1438,1440,1442,1444,1446,1449,1451,1453,1455,1458],{"class":75,"line":438},[187,1439,1403],{"class":192},[187,1441,1406],{"class":196},[187,1443,1328],{"class":265},[187,1445,269],{"class":192},[187,1447,1448],{"class":224},"401",[187,1450,535],{"class":196},[187,1452,1418],{"class":265},[187,1454,269],{"class":192},[187,1456,1457],{"class":531},"\"bad signature\"",[187,1459,275],{"class":196},[187,1461,1462],{"class":75,"line":459},[187,1463,218],{"emptyLinePlaceholder":217},[187,1465,1466,1469,1471,1473],{"class":75,"line":487},[187,1467,1468],{"class":196},"    event ",[187,1470,269],{"class":192},[187,1472,1377],{"class":192},[187,1474,1475],{"class":196}," request.json()\n",[187,1477,1478,1481],{"class":75,"line":492},[187,1479,1480],{"class":192},"    await",[187,1482,1483],{"class":196}," store_raw_event(\n",[187,1485,1486,1489,1491,1494,1497],{"class":75,"line":497},[187,1487,1488],{"class":265},"        event_id",[187,1490,269],{"class":192},[187,1492,1493],{"class":196},"event[",[187,1495,1496],{"class":531},"\"id\"",[187,1498,1499],{"class":196},"],\n",[187,1501,1502,1505,1507,1509,1512],{"class":75,"line":509},[187,1503,1504],{"class":265},"        event_type",[187,1506,269],{"class":192},[187,1508,1493],{"class":196},[187,1510,1511],{"class":531},"\"type\"",[187,1513,1499],{"class":196},[187,1515,1516,1519,1521],{"class":75,"line":518},[187,1517,1518],{"class":265},"        payload",[187,1520,269],{"class":192},[187,1522,1523],{"class":196},"payload,\n",[187,1525,1526],{"class":75,"line":544},[187,1527,635],{"class":196},[187,1529,1530,1532,1535,1537,1539,1541],{"class":75,"line":566},[187,1531,512],{"class":192},[187,1533,1534],{"class":196}," Response(",[187,1536,1328],{"class":265},[187,1538,269],{"class":192},[187,1540,1333],{"class":224},[187,1542,275],{"class":196},[14,1544,1545,1548,1549,1553,1554,1558,1559,1563],{},[184,1546,1547],{},"store_raw_event"," writes to a table with a unique constraint on the provider's event id and swallows the conflict. Providers deliver at least once, not exactly once, so duplicates are normal traffic rather than an error condition — the full pattern lives in ",[18,1550,1552],{"href":1551},"\u002Fautomating-side-hustle-operations-with-apis\u002Fprocessing-webhooks-with-python\u002Fbuilding-an-idempotent-webhook-receiver\u002F","building an idempotent webhook receiver",", and the provider-specific details of signature checking are covered in ",[18,1555,1557],{"href":1556},"\u002Fautomating-side-hustle-operations-with-apis\u002Fprocessing-webhooks-with-python\u002Fverifying-stripe-webhook-signatures\u002F","verifying Stripe webhook signatures",". Draining the stored events into your domain model belongs in a worker; ",[18,1560,1562],{"href":1561},"\u002Fscaling-and-operating-production-python-apis\u002Frunning-background-jobs-with-celery\u002F","running background jobs with Celery"," covers the queue side.",[27,1565,1567],{"id":1566},"hybrid-reconciliation-is-the-design-that-survives","Hybrid Reconciliation Is the Design That Survives",[14,1569,1570],{},"Webhooks fail silently. Your endpoint returns a 500 during a deploy, the provider retries for six hours and gives up, and nobody notices until a customer emails about a missing order three weeks later. A webhook-only architecture has no mechanism for discovering what it never received.",[14,1572,1573],{},"So run both. Webhooks carry the latency requirement; a slow, cheap sweep carries the correctness requirement. Once an hour — or nightly, depending on how much a missed record costs you — pull everything the provider changed since your last watermark and reconcile it against your local state. This sweep runs at a fraction of a percent of full polling cost because it fires once per account per interval, not once per account per minute.",[41,1575,50,1579,50,1582,50,1585,50,1587,50,1590,50,1594,50,1597,50,1601,50,1604,50,1608,50,1610,50,1613,50,1616,50,1619,50,1625,50,1627,50,1630,50,1634,50,1637,50,1641,50,1646,50,1649,50,1653,50,1656],{"viewBox":945,"role":44,"ariaLabelledBy":1576,"xmlns":48,"style":49},[1577,1578],"wvp-recon-t","wvp-recon-d",[52,1580,1581],{"id":1577},"Webhook stream with a dropped delivery recovered by the nightly sweep",[56,1583,1584],{"id":1578},"Five webhook deliveries land through the day, one fails during a deploy window, and the reconciliation sweep at 03:00 pulls changes since the watermark and backfills the missing record.",[60,1586],{"x":62,"y":62,"width":63,"height":957,"fill":65},[67,1588,1589],{"x":69,"y":960,"fill":71,"style":72},"One dropped delivery, one sweep, zero lost records",[75,1591],{"x1":1592,"y1":1059,"x2":1593,"y2":1059,"stroke":80,"style":81},"60","670",[67,1595,1596],{"x":1592,"y":1109,"fill":99,"style":158},"00:00",[67,1598,1600],{"x":1599,"y":1109,"fill":99,"style":158},"365","12:00",[67,1602,1603],{"x":1593,"y":1109,"fill":99,"style":158},"24:00",[1605,1606],"circle",{"cx":1049,"cy":1059,"r":1607,"fill":88,"stroke":146,"style":81},"7",[1605,1609],{"cx":957,"cy":1059,"r":1607,"fill":88,"stroke":146,"style":81},[1605,1611],{"cx":1612,"cy":1059,"r":1607,"fill":88,"stroke":146,"style":81},"430",[1605,1614],{"cx":1615,"cy":1059,"r":1607,"fill":88,"stroke":146,"style":81},"590",[1605,1617],{"cx":1618,"cy":1059,"r":1051,"fill":88,"stroke":89,"style":81},"330",[75,1620],{"x1":1621,"y1":1622,"x2":1623,"y2":1624,"stroke":89,"style":81},"325","145","335","155",[75,1626],{"x1":1623,"y1":1622,"x2":1621,"y2":1624,"stroke":89,"style":81},[67,1628,1629],{"x":1618,"y":105,"fill":71,"style":158},"500 during deploy",[67,1631,1633],{"x":1632,"y":105,"fill":99,"style":158},"190","delivered",[60,1635],{"x":1059,"y":1128,"width":1100,"height":1636,"rx":1051,"fill":88,"stroke":1052,"style":81},"34",[67,1638,1640],{"x":1639,"y":150,"fill":71,"style":94},"210","03:00 sweep",[75,1642],{"x1":1643,"y1":1644,"x2":1645,"y2":1644,"stroke":1052,"style":81},"270","213","322",[980,1647],{"d":1648,"fill":1052},"M322 213 L312 208 L312 218 z",[75,1650],{"x1":1618,"y1":1651,"x2":1618,"y2":1652,"stroke":1052,"style":81},"205","162",[980,1654],{"d":1655,"fill":1052},"M330 162 L325 172 L335 172 z",[67,1657,1659],{"x":1658,"y":150,"fill":71,"style":94},"470","backfills the gap from the watermark",[14,1661,1662,1663,1667,1668,175],{},"The sweep is short. Page through changed records since your stored watermark, upsert each one through the same handler the webhook path uses, then advance the watermark only after the page commits. Use ",[18,1664,1666],{"href":1665},"\u002Fgetting-started-with-python-apis-for-builders\u002Fmaking-http-requests-with-requests-library\u002Fhttpx-vs-requests-for-async\u002F","httpx for async work"," so a single process can reconcile many accounts concurrently, and schedule it through the patterns in ",[18,1669,1671],{"href":1670},"\u002Fautomating-side-hustle-operations-with-apis\u002Fscheduling-data-pipelines-with-cron\u002F","scheduling data pipelines with cron",[177,1673,1675],{"className":179,"code":1674,"language":181,"meta":182,"style":182},"import asyncio\nimport os\nfrom datetime import datetime, timedelta, timezone\n\nimport httpx\n\nAPI_BASE = os.environ[\"PROVIDER_API_BASE\"]\nAPI_TOKEN = os.environ[\"PROVIDER_API_TOKEN\"]\nPAGE_SIZE = int(os.getenv(\"SWEEP_PAGE_SIZE\", \"200\"))\nOVERLAP = timedelta(minutes=int(os.getenv(\"SWEEP_OVERLAP_MINUTES\", \"10\")))\n\n\nasync def sweep(client: httpx.AsyncClient, account_id: str) -> int:\n    watermark = await load_watermark(account_id)\n    cursor: str | None = None\n    seen = 0\n\n    while True:\n        params = {\n            \"account\": account_id,\n            \"updated_since\": (watermark - OVERLAP).isoformat(),\n            \"limit\": PAGE_SIZE,\n        }\n        if cursor:\n            params[\"cursor\"] = cursor\n\n        response = await client.get(f\"{API_BASE}\u002Fevents\", params=params)\n        response.raise_for_status()\n        body = response.json()\n\n        for record in body[\"data\"]:\n            await apply_event(record)\n            seen += 1\n\n        cursor = body.get(\"next_cursor\")\n        if not cursor:\n            break\n\n    await save_watermark(account_id, datetime.now(timezone.utc))\n    return seen\n\n\nasync def main() -> None:\n    headers = {\"Authorization\": f\"Bearer {API_TOKEN}\"}\n    limits = httpx.Limits(max_connections=int(os.getenv(\"SWEEP_CONCURRENCY\", \"20\")))\n    async with httpx.AsyncClient(headers=headers, timeout=30.0, limits=limits) as client:\n        accounts = await list_active_accounts()\n        results = await asyncio.gather(*(sweep(client, a) for a in accounts))\n    print(f\"reconciled {sum(results)} records across {len(accounts)} accounts\")\n\n\nif __name__ == \"__main__\":\n    asyncio.run(main())\n",[184,1676,1677,1684,1690,1702,1706,1713,1717,1732,1746,1767,1797,1801,1805,1825,1837,1855,1865,1869,1879,1889,1897,1914,1927,1932,1940,1956,1960,1992,1997,2007,2011,2031,2039,2050,2054,2069,2077,2082,2086,2093,2100,2104,2108,2125,2153,2182,2225,2237,2265,2300,2304,2308,2320],{"__ignoreMap":182},[187,1678,1679,1681],{"class":75,"line":189},[187,1680,193],{"class":192},[187,1682,1683],{"class":196}," asyncio\n",[187,1685,1686,1688],{"class":75,"line":200},[187,1687,193],{"class":192},[187,1689,197],{"class":196},[187,1691,1692,1694,1697,1699],{"class":75,"line":214},[187,1693,203],{"class":192},[187,1695,1696],{"class":196}," datetime ",[187,1698,193],{"class":192},[187,1700,1701],{"class":196}," datetime, timedelta, timezone\n",[187,1703,1704],{"class":75,"line":221},[187,1705,218],{"emptyLinePlaceholder":217},[187,1707,1708,1710],{"class":75,"line":245},[187,1709,193],{"class":192},[187,1711,1712],{"class":196}," httpx\n",[187,1714,1715],{"class":75,"line":250},[187,1716,218],{"emptyLinePlaceholder":217},[187,1718,1719,1722,1724,1726,1729],{"class":75,"line":255},[187,1720,1721],{"class":224},"API_BASE",[187,1723,228],{"class":192},[187,1725,1220],{"class":196},[187,1727,1728],{"class":531},"\"PROVIDER_API_BASE\"",[187,1730,1731],{"class":196},"]\n",[187,1733,1734,1737,1739,1741,1744],{"class":75,"line":278},[187,1735,1736],{"class":224},"API_TOKEN",[187,1738,228],{"class":192},[187,1740,1220],{"class":196},[187,1742,1743],{"class":531},"\"PROVIDER_API_TOKEN\"",[187,1745,1731],{"class":196},[187,1747,1748,1751,1753,1755,1757,1760,1762,1765],{"class":75,"line":290},[187,1749,1750],{"class":224},"PAGE_SIZE",[187,1752,228],{"class":192},[187,1754,1236],{"class":224},[187,1756,528],{"class":196},[187,1758,1759],{"class":531},"\"SWEEP_PAGE_SIZE\"",[187,1761,535],{"class":196},[187,1763,1764],{"class":531},"\"200\"",[187,1766,1249],{"class":196},[187,1768,1769,1772,1774,1777,1780,1782,1784,1786,1789,1791,1794],{"class":75,"line":299},[187,1770,1771],{"class":224},"OVERLAP",[187,1773,228],{"class":192},[187,1775,1776],{"class":196}," timedelta(",[187,1778,1779],{"class":265},"minutes",[187,1781,269],{"class":192},[187,1783,358],{"class":224},[187,1785,528],{"class":196},[187,1787,1788],{"class":531},"\"SWEEP_OVERLAP_MINUTES\"",[187,1790,535],{"class":196},[187,1792,1793],{"class":531},"\"10\"",[187,1795,1796],{"class":196},")))\n",[187,1798,1799],{"class":75,"line":307},[187,1800,218],{"emptyLinePlaceholder":217},[187,1802,1803],{"class":75,"line":315},[187,1804,218],{"emptyLinePlaceholder":217},[187,1806,1807,1809,1811,1814,1817,1819,1821,1823],{"class":75,"line":324},[187,1808,1340],{"class":192},[187,1810,1343],{"class":192},[187,1812,1813],{"class":258}," sweep",[187,1815,1816],{"class":196},"(client: httpx.AsyncClient, account_id: ",[187,1818,1276],{"class":224},[187,1820,451],{"class":196},[187,1822,358],{"class":224},[187,1824,287],{"class":196},[187,1826,1827,1830,1832,1834],{"class":75,"line":332},[187,1828,1829],{"class":196},"    watermark ",[187,1831,269],{"class":192},[187,1833,1377],{"class":192},[187,1835,1836],{"class":196}," load_watermark(account_id)\n",[187,1838,1839,1842,1844,1847,1850,1852],{"class":75,"line":337},[187,1840,1841],{"class":196},"    cursor: ",[187,1843,1276],{"class":224},[187,1845,1846],{"class":192}," |",[187,1848,1849],{"class":224}," None",[187,1851,228],{"class":192},[187,1853,1854],{"class":224}," None\n",[187,1856,1857,1860,1862],{"class":75,"line":346},[187,1858,1859],{"class":196},"    seen ",[187,1861,269],{"class":192},[187,1863,1864],{"class":224}," 0\n",[187,1866,1867],{"class":75,"line":363},[187,1868,218],{"emptyLinePlaceholder":217},[187,1870,1871,1874,1877],{"class":75,"line":391},[187,1872,1873],{"class":192},"    while",[187,1875,1876],{"class":224}," True",[187,1878,287],{"class":196},[187,1880,1881,1884,1886],{"class":75,"line":396},[187,1882,1883],{"class":196},"        params ",[187,1885,269],{"class":192},[187,1887,1888],{"class":196}," {\n",[187,1890,1891,1894],{"class":75,"line":403},[187,1892,1893],{"class":531},"            \"account\"",[187,1895,1896],{"class":196},": account_id,\n",[187,1898,1899,1902,1905,1908,1911],{"class":75,"line":417},[187,1900,1901],{"class":531},"            \"updated_since\"",[187,1903,1904],{"class":196},": (watermark ",[187,1906,1907],{"class":192},"-",[187,1909,1910],{"class":224}," OVERLAP",[187,1912,1913],{"class":196},").isoformat(),\n",[187,1915,1916,1919,1922,1924],{"class":75,"line":433},[187,1917,1918],{"class":531},"            \"limit\"",[187,1920,1921],{"class":196},": ",[187,1923,1750],{"class":224},[187,1925,1926],{"class":196},",\n",[187,1928,1929],{"class":75,"line":438},[187,1930,1931],{"class":196},"        }\n",[187,1933,1934,1937],{"class":75,"line":459},[187,1935,1936],{"class":192},"        if",[187,1938,1939],{"class":196}," cursor:\n",[187,1941,1942,1945,1948,1951,1953],{"class":75,"line":487},[187,1943,1944],{"class":196},"            params[",[187,1946,1947],{"class":531},"\"cursor\"",[187,1949,1950],{"class":196},"] ",[187,1952,269],{"class":192},[187,1954,1955],{"class":196}," cursor\n",[187,1957,1958],{"class":75,"line":492},[187,1959,218],{"emptyLinePlaceholder":217},[187,1961,1962,1965,1967,1969,1972,1974,1976,1979,1982,1984,1987,1989],{"class":75,"line":497},[187,1963,1964],{"class":196},"        response ",[187,1966,269],{"class":192},[187,1968,1377],{"class":192},[187,1970,1971],{"class":196}," client.get(",[187,1973,827],{"class":192},[187,1975,858],{"class":531},[187,1977,1978],{"class":224},"{API_BASE}",[187,1980,1981],{"class":531},"\u002Fevents\"",[187,1983,535],{"class":196},[187,1985,1986],{"class":265},"params",[187,1988,269],{"class":192},[187,1990,1991],{"class":196},"params)\n",[187,1993,1994],{"class":75,"line":509},[187,1995,1996],{"class":196},"        response.raise_for_status()\n",[187,1998,1999,2002,2004],{"class":75,"line":518},[187,2000,2001],{"class":196},"        body ",[187,2003,269],{"class":192},[187,2005,2006],{"class":196}," response.json()\n",[187,2008,2009],{"class":75,"line":544},[187,2010,218],{"emptyLinePlaceholder":217},[187,2012,2013,2016,2019,2022,2025,2028],{"class":75,"line":566},[187,2014,2015],{"class":192},"        for",[187,2017,2018],{"class":196}," record ",[187,2020,2021],{"class":192},"in",[187,2023,2024],{"class":196}," body[",[187,2026,2027],{"class":531},"\"data\"",[187,2029,2030],{"class":196},"]:\n",[187,2032,2033,2036],{"class":75,"line":588},[187,2034,2035],{"class":192},"            await",[187,2037,2038],{"class":196}," apply_event(record)\n",[187,2040,2041,2044,2047],{"class":75,"line":610},[187,2042,2043],{"class":196},"            seen ",[187,2045,2046],{"class":192},"+=",[187,2048,2049],{"class":224}," 1\n",[187,2051,2052],{"class":75,"line":632},[187,2053,218],{"emptyLinePlaceholder":217},[187,2055,2056,2059,2061,2064,2067],{"class":75,"line":638},[187,2057,2058],{"class":196},"        cursor ",[187,2060,269],{"class":192},[187,2062,2063],{"class":196}," body.get(",[187,2065,2066],{"class":531},"\"next_cursor\"",[187,2068,275],{"class":196},[187,2070,2071,2073,2075],{"class":75,"line":643},[187,2072,1936],{"class":192},[187,2074,1432],{"class":192},[187,2076,1939],{"class":196},[187,2078,2079],{"class":75,"line":648},[187,2080,2081],{"class":192},"            break\n",[187,2083,2084],{"class":75,"line":665},[187,2085,218],{"emptyLinePlaceholder":217},[187,2087,2088,2090],{"class":75,"line":676},[187,2089,1480],{"class":192},[187,2091,2092],{"class":196}," save_watermark(account_id, datetime.now(timezone.utc))\n",[187,2094,2095,2097],{"class":75,"line":687},[187,2096,512],{"class":192},[187,2098,2099],{"class":196}," seen\n",[187,2101,2102],{"class":75,"line":698},[187,2103,218],{"emptyLinePlaceholder":217},[187,2105,2106],{"class":75,"line":722},[187,2107,218],{"emptyLinePlaceholder":217},[187,2109,2110,2112,2114,2117,2120,2123],{"class":75,"line":727},[187,2111,1340],{"class":192},[187,2113,1343],{"class":192},[187,2115,2116],{"class":258}," main",[187,2118,2119],{"class":196},"() -> ",[187,2121,2122],{"class":224},"None",[187,2124,287],{"class":196},[187,2126,2127,2130,2132,2135,2138,2140,2142,2145,2148,2150],{"class":75,"line":736},[187,2128,2129],{"class":196},"    headers ",[187,2131,269],{"class":192},[187,2133,2134],{"class":196}," {",[187,2136,2137],{"class":531},"\"Authorization\"",[187,2139,1921],{"class":196},[187,2141,827],{"class":192},[187,2143,2144],{"class":531},"\"Bearer ",[187,2146,2147],{"class":224},"{API_TOKEN}",[187,2149,858],{"class":531},[187,2151,2152],{"class":196},"}\n",[187,2154,2155,2158,2160,2163,2166,2168,2170,2172,2175,2177,2180],{"class":75,"line":757},[187,2156,2157],{"class":196},"    limits ",[187,2159,269],{"class":192},[187,2161,2162],{"class":196}," httpx.Limits(",[187,2164,2165],{"class":265},"max_connections",[187,2167,269],{"class":192},[187,2169,358],{"class":224},[187,2171,528],{"class":196},[187,2173,2174],{"class":531},"\"SWEEP_CONCURRENCY\"",[187,2176,535],{"class":196},[187,2178,2179],{"class":531},"\"20\"",[187,2181,1796],{"class":196},[187,2183,2184,2187,2190,2193,2196,2198,2201,2204,2206,2209,2211,2214,2216,2219,2222],{"class":75,"line":768},[187,2185,2186],{"class":192},"    async",[187,2188,2189],{"class":192}," with",[187,2191,2192],{"class":196}," httpx.AsyncClient(",[187,2194,2195],{"class":265},"headers",[187,2197,269],{"class":192},[187,2199,2200],{"class":196},"headers, ",[187,2202,2203],{"class":265},"timeout",[187,2205,269],{"class":192},[187,2207,2208],{"class":224},"30.0",[187,2210,535],{"class":196},[187,2212,2213],{"class":265},"limits",[187,2215,269],{"class":192},[187,2217,2218],{"class":196},"limits) ",[187,2220,2221],{"class":192},"as",[187,2223,2224],{"class":196}," client:\n",[187,2226,2227,2230,2232,2234],{"class":75,"line":786},[187,2228,2229],{"class":196},"        accounts ",[187,2231,269],{"class":192},[187,2233,1377],{"class":192},[187,2235,2236],{"class":196}," list_active_accounts()\n",[187,2238,2239,2242,2244,2246,2249,2251,2254,2257,2260,2262],{"class":75,"line":796},[187,2240,2241],{"class":196},"        results ",[187,2243,269],{"class":192},[187,2245,1377],{"class":192},[187,2247,2248],{"class":196}," asyncio.gather(",[187,2250,375],{"class":192},[187,2252,2253],{"class":196},"(sweep(client, a) ",[187,2255,2256],{"class":192},"for",[187,2258,2259],{"class":196}," a ",[187,2261,2021],{"class":192},[187,2263,2264],{"class":196}," accounts))\n",[187,2266,2267,2269,2271,2273,2276,2279,2282,2284,2287,2290,2293,2295,2298],{"class":75,"line":804},[187,2268,822],{"class":224},[187,2270,262],{"class":196},[187,2272,827],{"class":192},[187,2274,2275],{"class":531},"\"reconciled ",[187,2277,2278],{"class":224},"{sum",[187,2280,2281],{"class":196},"(results)",[187,2283,842],{"class":224},[187,2285,2286],{"class":531}," records across ",[187,2288,2289],{"class":224},"{len",[187,2291,2292],{"class":196},"(accounts)",[187,2294,842],{"class":224},[187,2296,2297],{"class":531}," accounts\"",[187,2299,275],{"class":196},[187,2301,2302],{"class":75,"line":814},[187,2303,218],{"emptyLinePlaceholder":217},[187,2305,2306],{"class":75,"line":819},[187,2307,218],{"emptyLinePlaceholder":217},[187,2309,2310,2312,2314,2316,2318],{"class":75,"line":863},[187,2311,651],{"class":192},[187,2313,654],{"class":224},[187,2315,657],{"class":192},[187,2317,660],{"class":531},[187,2319,287],{"class":196},[187,2321,2322],{"class":75,"line":899},[187,2323,2324],{"class":196},"    asyncio.run(main())\n",[14,2326,2327,2328,2331],{},"Two details make this reliable. The overlap window re-reads a few minutes you already processed, which costs nothing because ",[184,2329,2330],{},"apply_event"," is idempotent and protects you from clock skew between your box and theirs. And advancing the watermark only after a successful page means a crash mid-sweep replays rather than skips.",[27,2333,2335],{"id":2334},"when-the-third-party-offers-no-webhook","When the Third Party Offers No Webhook",[14,2337,2338],{},"Plenty of APIs never send you anything. You still have better options than a naive fixed-interval loop, and they can cut effective request cost by 80% or more.",[41,2340,50,2345,50,2348,50,2351,50,2354,50,2358,50,2362,50,2366,50,2370,50,2372,50,2376,50,2378,50,2381,50,2384,50,2388,50,2390,50,2393,50,2396,50,2400,50,2402,50,2405,50,2410,50,2413,50,2415,50,2418,50,2421,50,2424,50,2426,50,2429,50,2433,50,2436,50,2439,50,2443,50,2446,50,2449,50,2451,50,2456,50,2459],{"viewBox":2341,"role":44,"ariaLabelledBy":2342,"xmlns":48,"style":49},"0 0 720 340",[2343,2344],"wvp-tree-t","wvp-tree-d",[52,2346,2347],{"id":2343},"Decision path when choosing a change-detection strategy",[56,2349,2350],{"id":2344},"Check for webhooks first, then ETag or Last-Modified support, then an updated-since cursor, and fall back to an adaptive interval with body hashing.",[60,2352],{"x":62,"y":62,"width":63,"height":2353,"fill":65},"340",[60,2355],{"x":2356,"y":960,"width":957,"height":2357,"rx":1051,"fill":88,"stroke":146,"style":81},"30","44",[67,2359,2361],{"x":1624,"y":2360,"fill":71,"style":94},"51","Webhooks offered?",[60,2363],{"x":2364,"y":960,"width":2365,"height":2357,"rx":1051,"fill":88,"stroke":146,"style":81},"400","290",[67,2367,2369],{"x":2368,"y":2360,"fill":71,"style":94},"545","Webhook plus nightly sweep",[60,2371],{"x":2356,"y":118,"width":957,"height":2357,"rx":1051,"fill":88,"stroke":1052,"style":81},[67,2373,2375],{"x":1624,"y":2374,"fill":71,"style":94},"131","ETag or Last-Modified?",[60,2377],{"x":2364,"y":118,"width":2365,"height":2357,"rx":1051,"fill":88,"stroke":1052,"style":81},[67,2379,2380],{"x":2368,"y":2374,"fill":71,"style":94},"Conditional GET, 304 is cheap",[60,2382],{"x":2356,"y":2383,"width":957,"height":2357,"rx":1051,"fill":88,"stroke":119,"style":81},"184",[67,2385,2387],{"x":1624,"y":2386,"fill":71,"style":94},"211","updated_since cursor?",[60,2389],{"x":2364,"y":2383,"width":2365,"height":2357,"rx":1051,"fill":88,"stroke":119,"style":81},[67,2391,2392],{"x":2368,"y":2386,"fill":71,"style":94},"Cursor poll, page forward only",[60,2394],{"x":2356,"y":2395,"width":957,"height":2357,"rx":1051,"fill":88,"stroke":89,"style":81},"264",[67,2397,2399],{"x":1624,"y":2398,"fill":71,"style":94},"291","Nothing but plain GET",[60,2401],{"x":2364,"y":2395,"width":2365,"height":2357,"rx":1051,"fill":88,"stroke":89,"style":81},[67,2403,2404],{"x":2368,"y":2398,"fill":71,"style":94},"Adaptive interval plus body hash",[75,2406],{"x1":2407,"y1":2408,"x2":2409,"y2":2408,"stroke":99,"style":81},"280","46","398",[980,2411],{"d":2412,"fill":99},"M398 46 L388 41 L388 51 z",[75,2414],{"x1":2407,"y1":105,"x2":2409,"y2":105,"stroke":99,"style":81},[980,2416],{"d":2417,"fill":99},"M398 126 L388 121 L388 131 z",[75,2419],{"x1":2407,"y1":2420,"x2":2409,"y2":2420,"stroke":99,"style":81},"206",[980,2422],{"d":2423,"fill":99},"M398 206 L388 201 L388 211 z",[75,2425],{"x1":2407,"y1":157,"x2":2409,"y2":157,"stroke":99,"style":81},[980,2427],{"d":2428,"fill":99},"M398 286 L388 281 L388 291 z",[67,2430,2432],{"x":2431,"y":1050,"fill":99,"style":158},"339","yes",[67,2434,2432],{"x":2431,"y":2435,"fill":99,"style":158},"118",[67,2437,2432],{"x":2431,"y":2438,"fill":99,"style":158},"198",[67,2440,2442],{"x":2431,"y":2441,"fill":99,"style":158},"278","else",[75,2444],{"x1":1592,"y1":2445,"x2":1592,"y2":118,"stroke":99,"style":81},"68",[75,2447],{"x1":1592,"y1":2448,"x2":1592,"y2":2383,"stroke":99,"style":81},"148",[75,2450],{"x1":1592,"y1":992,"x2":1592,"y2":2395,"stroke":99,"style":81},[67,2452,2455],{"x":2453,"y":92,"fill":99,"style":2454},"88","text-anchor:start;font-size:11;font-family:var(--font-sans);","no",[67,2457,2455],{"x":2453,"y":2458,"fill":99,"style":2454},"170",[67,2460,2455],{"x":2453,"y":957,"fill":99,"style":2454},[14,2462,2463,2464,2467,2468,2471],{},"Conditional requests are the biggest single win. Send ",[184,2465,2466],{},"If-None-Match"," with the last ",[184,2469,2470],{},"ETag"," you saw; an unchanged resource returns a 304 with an empty body. You still pay a round trip and a rate-limit slot, but you skip the payload transfer and all deserialisation, which is where most of your CPU time goes. Layer adaptive intervals on top: back off when nothing changes, tighten up immediately after a change, because activity arrives in bursts.",[177,2473,2475],{"className":179,"code":2474,"language":181,"meta":182,"style":182},"import asyncio\nimport hashlib\nimport os\n\nimport httpx\n\nRESOURCE_URL = os.environ[\"WATCHED_RESOURCE_URL\"]\nMIN_INTERVAL = float(os.getenv(\"POLL_MIN_SECONDS\", \"15\"))\nMAX_INTERVAL = float(os.getenv(\"POLL_MAX_SECONDS\", \"900\"))\nBACKOFF_FACTOR = float(os.getenv(\"POLL_BACKOFF_FACTOR\", \"1.6\"))\n\n\nasync def watch(client: httpx.AsyncClient) -> None:\n    interval = MIN_INTERVAL\n    etag: str | None = None\n    digest: str | None = None\n\n    while True:\n        headers = {\"If-None-Match\": etag} if etag else {}\n        response = await client.get(RESOURCE_URL, headers=headers)\n\n        match response.status_code:\n            case 304:\n                changed = False\n            case 200:\n                etag = response.headers.get(\"etag\")\n                new_digest = hashlib.sha256(response.content).hexdigest()\n                changed = new_digest != digest\n                digest = new_digest\n                if changed:\n                    await apply_snapshot(response.json())\n            case 429:\n                retry_after = float(response.headers.get(\"retry-after\", MAX_INTERVAL))\n                await asyncio.sleep(retry_after)\n                continue\n            case _:\n                response.raise_for_status()\n                changed = False\n\n        interval = MIN_INTERVAL if changed else min(interval * BACKOFF_FACTOR, MAX_INTERVAL)\n        await asyncio.sleep(interval)\n",[184,2476,2477,2483,2489,2495,2499,2505,2509,2523,2545,2566,2587,2591,2595,2611,2621,2636,2651,2655,2663,2688,2709,2713,2721,2731,2741,2750,2765,2775,2790,2800,2808,2816,2825,2846,2854,2859,2865,2870,2878,2882,2917],{"__ignoreMap":182},[187,2478,2479,2481],{"class":75,"line":189},[187,2480,193],{"class":192},[187,2482,1683],{"class":196},[187,2484,2485,2487],{"class":75,"line":200},[187,2486,193],{"class":192},[187,2488,1167],{"class":196},[187,2490,2491,2493],{"class":75,"line":214},[187,2492,193],{"class":192},[187,2494,197],{"class":196},[187,2496,2497],{"class":75,"line":221},[187,2498,218],{"emptyLinePlaceholder":217},[187,2500,2501,2503],{"class":75,"line":245},[187,2502,193],{"class":192},[187,2504,1712],{"class":196},[187,2506,2507],{"class":75,"line":250},[187,2508,218],{"emptyLinePlaceholder":217},[187,2510,2511,2514,2516,2518,2521],{"class":75,"line":255},[187,2512,2513],{"class":224},"RESOURCE_URL",[187,2515,228],{"class":192},[187,2517,1220],{"class":196},[187,2519,2520],{"class":531},"\"WATCHED_RESOURCE_URL\"",[187,2522,1731],{"class":196},[187,2524,2525,2528,2530,2533,2535,2538,2540,2543],{"class":75,"line":278},[187,2526,2527],{"class":224},"MIN_INTERVAL",[187,2529,228],{"class":192},[187,2531,2532],{"class":224}," float",[187,2534,528],{"class":196},[187,2536,2537],{"class":531},"\"POLL_MIN_SECONDS\"",[187,2539,535],{"class":196},[187,2541,2542],{"class":531},"\"15\"",[187,2544,1249],{"class":196},[187,2546,2547,2550,2552,2554,2556,2559,2561,2564],{"class":75,"line":290},[187,2548,2549],{"class":224},"MAX_INTERVAL",[187,2551,228],{"class":192},[187,2553,2532],{"class":224},[187,2555,528],{"class":196},[187,2557,2558],{"class":531},"\"POLL_MAX_SECONDS\"",[187,2560,535],{"class":196},[187,2562,2563],{"class":531},"\"900\"",[187,2565,1249],{"class":196},[187,2567,2568,2571,2573,2575,2577,2580,2582,2585],{"class":75,"line":299},[187,2569,2570],{"class":224},"BACKOFF_FACTOR",[187,2572,228],{"class":192},[187,2574,2532],{"class":224},[187,2576,528],{"class":196},[187,2578,2579],{"class":531},"\"POLL_BACKOFF_FACTOR\"",[187,2581,535],{"class":196},[187,2583,2584],{"class":531},"\"1.6\"",[187,2586,1249],{"class":196},[187,2588,2589],{"class":75,"line":307},[187,2590,218],{"emptyLinePlaceholder":217},[187,2592,2593],{"class":75,"line":315},[187,2594,218],{"emptyLinePlaceholder":217},[187,2596,2597,2599,2601,2604,2607,2609],{"class":75,"line":324},[187,2598,1340],{"class":192},[187,2600,1343],{"class":192},[187,2602,2603],{"class":258}," watch",[187,2605,2606],{"class":196},"(client: httpx.AsyncClient) -> ",[187,2608,2122],{"class":224},[187,2610,287],{"class":196},[187,2612,2613,2616,2618],{"class":75,"line":332},[187,2614,2615],{"class":196},"    interval ",[187,2617,269],{"class":192},[187,2619,2620],{"class":224}," MIN_INTERVAL\n",[187,2622,2623,2626,2628,2630,2632,2634],{"class":75,"line":337},[187,2624,2625],{"class":196},"    etag: ",[187,2627,1276],{"class":224},[187,2629,1846],{"class":192},[187,2631,1849],{"class":224},[187,2633,228],{"class":192},[187,2635,1854],{"class":224},[187,2637,2638,2641,2643,2645,2647,2649],{"class":75,"line":346},[187,2639,2640],{"class":196},"    digest: ",[187,2642,1276],{"class":224},[187,2644,1846],{"class":192},[187,2646,1849],{"class":224},[187,2648,228],{"class":192},[187,2650,1854],{"class":224},[187,2652,2653],{"class":75,"line":363},[187,2654,218],{"emptyLinePlaceholder":217},[187,2656,2657,2659,2661],{"class":75,"line":391},[187,2658,1873],{"class":192},[187,2660,1876],{"class":224},[187,2662,287],{"class":196},[187,2664,2665,2668,2670,2672,2675,2678,2680,2683,2685],{"class":75,"line":396},[187,2666,2667],{"class":196},"        headers ",[187,2669,269],{"class":192},[187,2671,2134],{"class":196},[187,2673,2674],{"class":531},"\"If-None-Match\"",[187,2676,2677],{"class":196},": etag} ",[187,2679,651],{"class":192},[187,2681,2682],{"class":196}," etag ",[187,2684,2442],{"class":192},[187,2686,2687],{"class":196}," {}\n",[187,2689,2690,2692,2694,2696,2698,2700,2702,2704,2706],{"class":75,"line":403},[187,2691,1964],{"class":196},[187,2693,269],{"class":192},[187,2695,1377],{"class":192},[187,2697,1971],{"class":196},[187,2699,2513],{"class":224},[187,2701,535],{"class":196},[187,2703,2195],{"class":265},[187,2705,269],{"class":192},[187,2707,2708],{"class":196},"headers)\n",[187,2710,2711],{"class":75,"line":417},[187,2712,218],{"emptyLinePlaceholder":217},[187,2714,2715,2718],{"class":75,"line":433},[187,2716,2717],{"class":192},"        match",[187,2719,2720],{"class":196}," response.status_code:\n",[187,2722,2723,2726,2729],{"class":75,"line":438},[187,2724,2725],{"class":192},"            case",[187,2727,2728],{"class":224}," 304",[187,2730,287],{"class":196},[187,2732,2733,2736,2738],{"class":75,"line":459},[187,2734,2735],{"class":196},"                changed ",[187,2737,269],{"class":192},[187,2739,2740],{"class":224}," False\n",[187,2742,2743,2745,2748],{"class":75,"line":487},[187,2744,2725],{"class":192},[187,2746,2747],{"class":224}," 200",[187,2749,287],{"class":196},[187,2751,2752,2755,2757,2760,2763],{"class":75,"line":492},[187,2753,2754],{"class":196},"                etag ",[187,2756,269],{"class":192},[187,2758,2759],{"class":196}," response.headers.get(",[187,2761,2762],{"class":531},"\"etag\"",[187,2764,275],{"class":196},[187,2766,2767,2770,2772],{"class":75,"line":497},[187,2768,2769],{"class":196},"                new_digest ",[187,2771,269],{"class":192},[187,2773,2774],{"class":196}," hashlib.sha256(response.content).hexdigest()\n",[187,2776,2777,2779,2781,2784,2787],{"class":75,"line":509},[187,2778,2735],{"class":196},[187,2780,269],{"class":192},[187,2782,2783],{"class":196}," new_digest ",[187,2785,2786],{"class":192},"!=",[187,2788,2789],{"class":196}," digest\n",[187,2791,2792,2795,2797],{"class":75,"line":518},[187,2793,2794],{"class":196},"                digest ",[187,2796,269],{"class":192},[187,2798,2799],{"class":196}," new_digest\n",[187,2801,2802,2805],{"class":75,"line":544},[187,2803,2804],{"class":192},"                if",[187,2806,2807],{"class":196}," changed:\n",[187,2809,2810,2813],{"class":75,"line":566},[187,2811,2812],{"class":192},"                    await",[187,2814,2815],{"class":196}," apply_snapshot(response.json())\n",[187,2817,2818,2820,2823],{"class":75,"line":588},[187,2819,2725],{"class":192},[187,2821,2822],{"class":224}," 429",[187,2824,287],{"class":196},[187,2826,2827,2830,2832,2834,2837,2840,2842,2844],{"class":75,"line":610},[187,2828,2829],{"class":196},"                retry_after ",[187,2831,269],{"class":192},[187,2833,2532],{"class":224},[187,2835,2836],{"class":196},"(response.headers.get(",[187,2838,2839],{"class":531},"\"retry-after\"",[187,2841,535],{"class":196},[187,2843,2549],{"class":224},[187,2845,1249],{"class":196},[187,2847,2848,2851],{"class":75,"line":632},[187,2849,2850],{"class":192},"                await",[187,2852,2853],{"class":196}," asyncio.sleep(retry_after)\n",[187,2855,2856],{"class":75,"line":638},[187,2857,2858],{"class":192},"                continue\n",[187,2860,2861,2863],{"class":75,"line":643},[187,2862,2725],{"class":192},[187,2864,801],{"class":196},[187,2866,2867],{"class":75,"line":648},[187,2868,2869],{"class":196},"                response.raise_for_status()\n",[187,2871,2872,2874,2876],{"class":75,"line":665},[187,2873,2735],{"class":196},[187,2875,269],{"class":192},[187,2877,2740],{"class":224},[187,2879,2880],{"class":75,"line":676},[187,2881,218],{"emptyLinePlaceholder":217},[187,2883,2884,2887,2889,2892,2895,2898,2900,2903,2906,2908,2911,2913,2915],{"class":75,"line":687},[187,2885,2886],{"class":196},"        interval ",[187,2888,269],{"class":192},[187,2890,2891],{"class":224}," MIN_INTERVAL",[187,2893,2894],{"class":192}," if",[187,2896,2897],{"class":196}," changed ",[187,2899,2442],{"class":192},[187,2901,2902],{"class":224}," min",[187,2904,2905],{"class":196},"(interval ",[187,2907,375],{"class":192},[187,2909,2910],{"class":224}," BACKOFF_FACTOR",[187,2912,535],{"class":196},[187,2914,2549],{"class":224},[187,2916,275],{"class":196},[187,2918,2919,2922],{"class":75,"line":698},[187,2920,2921],{"class":192},"        await",[187,2923,2924],{"class":196}," asyncio.sleep(interval)\n",[14,2926,2927,2928,2932,2933,2937],{},"The body hash matters because some providers rotate weak ETags on every response even when nothing changed. Hashing the payload yourself means you only pay for downstream writes when the content genuinely differs. Cache the last digest in Redis rather than process memory so a restart does not trigger a full re-sync — see ",[18,2929,2931],{"href":2930},"\u002Fscaling-and-operating-production-python-apis\u002Fcaching-python-api-responses-with-redis\u002F","caching Python API responses with Redis",". If the provider has no usable API at all, the trade-offs shift entirely and ",[18,2934,2936],{"href":2935},"\u002Fautomating-side-hustle-operations-with-apis\u002Fweb-scraping-vs-official-apis\u002F","web scraping vs official APIs"," is the honest next stop.",[27,2939,2941],{"id":2940},"when-to-choose-each","When to Choose Each",[2943,2944,2945,2961],"table",{},[2946,2947,2948],"thead",{},[2949,2950,2951,2955,2958],"tr",{},[2952,2953,2954],"th",{},"Situation",[2952,2956,2957],{},"Pick",[2952,2959,2960],{},"Why",[2962,2963,2964,2976,2987,2998,3009],"tbody",{},[2949,2965,2966,2970,2973],{},[2967,2968,2969],"td",{},"Payment or signup events",[2967,2971,2972],{},"Webhook + sweep",[2967,2974,2975],{},"Seconds matter, volume is low",[2949,2977,2978,2981,2984],{},[2967,2979,2980],{},"Under 20 watched resources",[2967,2982,2983],{},"Polling",[2967,2985,2986],{},"Ops cost beats compute saved",[2949,2988,2989,2992,2995],{},[2967,2990,2991],{},"No public HTTPS endpoint yet",[2967,2993,2994],{},"Conditional polling",[2967,2996,2997],{},"Ship now, migrate later",[2949,2999,3000,3003,3006],{},[2967,3001,3002],{},"Bursty, high-volume feeds",[2967,3004,3005],{},"Polling batches",[2967,3007,3008],{},"One call returns many changes",[2949,3010,3011,3014,3017],{},[2967,3012,3013],{},"Regulated or audited data",[2967,3015,3016],{},"Webhook + hourly sweep",[2967,3018,3019],{},"Sweep is your proof of completeness",[14,3021,3022],{},"Avoid webhooks while you are still on a laptop with no stable public URL, when the provider's retry policy is weaker than your uptime, or when you cannot yet run a queue. Avoid polling once you cross roughly 50 watched resources at sub-five-minute freshness — that is where the vCPU bill and the rate-limit fights start compounding.",[27,3024,3026],{"id":3025},"migration-path-from-polling-to-webhooks","Migration Path from Polling to Webhooks",[14,3028,3029,3030,3034],{},"Switching is four steps and none of them require downtime. First, deploy the receiver and store raw events without acting on them, while the poller keeps running as the source of truth. Second, compare for a week: log every event id the webhook saw and every change the poller found, and alert on divergence. You want the webhook set to be a superset. Third, flip the write path so the queue worker applies events and the poller only reconciles. Fourth, stretch the poll interval from one minute to one hour and watch your request graph fall off a cliff. Keep the sweep forever. Instrument each stage through ",[18,3031,3033],{"href":3032},"\u002Fscaling-and-operating-production-python-apis\u002Fmonitoring-and-logging-python-apis\u002F","monitoring and logging Python APIs"," so the divergence alert is real rather than aspirational.",[27,3036,3038],{"id":3037},"builder-verdict","Builder Verdict",[14,3040,3041],{},"Webhooks win, and it is not close. For any integration where resources change less often than your freshness target — payments, CRM updates, document processing, order sync — a signed receiver plus an hourly reconciliation sweep costs a rounding error compared to polling, delivers single-digit-second latency instead of minutes, and stops you fighting rate limits at exactly the moment your customer count grows. The one honest cost is a day of extra work: signature verification, an idempotent write path, and a queue. Pay it. Polling is the right call only in two narrow cases — fewer than twenty watched resources, where operational simplicity genuinely beats compute savings, and genuinely high-frequency feeds where one request batches dozens of changes. Everywhere else, keep polling only as the slow sweep that proves your webhook stream did not lie to you.",[27,3043,3045],{"id":3044},"faq","FAQ",[14,3047,3048,3051],{},[164,3049,3050],{},"At what customer count does polling stop being profitable?","\nAround 50 connected accounts at one-minute freshness. That is 2.16 million outbound requests a month and roughly 72 CPU-hours, which is where a hobby-tier instance starts throttling and you upgrade. Below 20 accounts the compute is genuinely free and the extra webhook infrastructure is not worth your time.",[14,3053,3054,3057],{},[164,3055,3056],{},"Does a reconciliation sweep undo the cost savings of webhooks?","\nNo. An hourly sweep across 500 accounts is 360,000 requests a month against 21.6 million for one-minute polling — a 98% reduction while still guaranteeing completeness. Move it to nightly and it drops to 15,000. The sweep is insurance you can afford.",[14,3059,3060,3063],{},[164,3061,3062],{},"What happens to my webhook endpoint when I rotate the signing secret?","\nAccept both the old and new secret for one overlap window, typically 24 hours, by trying each in turn during verification. Register the new endpoint secret with the provider first, confirm live traffic verifies against it, then drop the old one. Never rotate in a single deploy — in-flight retries carry the old signature.",[14,3065,3066,3069],{},[164,3067,3068],{},"Is migrating from polling to webhooks risky for existing customers?","\nNot if you shadow first. Run the receiver in log-only mode alongside the poller for a week and compare the two event streams. Customers see nothing until the sets match, and you keep the poller as a sweep, so the worst regression is a return to your current latency rather than lost data.",[14,3071,3072,3075],{},[164,3073,3074],{},"How do I handle a provider that fires thousands of webhooks in one burst?","\nReturn 202 immediately and absorb the burst in the queue, never in the request handler. Size the queue for peak and the workers for average, since events are not time-critical once persisted. If the provider supports it, subscribe to only the event types you act on — dropping unused types often cuts inbound volume by half.",[27,3077,3079],{"id":3078},"related","Related",[3081,3082,3083,3089,3096,3102,3109],"ul",{},[3084,3085,3086,3088],"li",{},[18,3087,21],{"href":20}," — the parent guide, for choosing the protocol before you choose who initiates the call.",[3084,3090,3091,3095],{},[18,3092,3094],{"href":3093},"\u002Fautomating-side-hustle-operations-with-apis\u002Fprocessing-webhooks-with-python\u002F","Processing webhooks with Python"," — the full receiver, queue, and replay stack once you have committed to webhooks.",[3084,3097,3098,3101],{},[18,3099,3100],{"href":1551},"Building an idempotent webhook receiver"," — how to make duplicate deliveries harmless instead of corrupting data.",[3084,3103,3104,3108],{},[18,3105,3107],{"href":3106},"\u002Fgetting-started-with-python-apis-for-builders\u002Fmaking-http-requests-with-requests-library\u002Fretrying-failed-http-requests-with-tenacity\u002F","Retrying failed HTTP requests with tenacity"," — the retry policy your reconciliation sweep needs when the provider wobbles.",[3084,3110,3111,3115],{},[18,3112,3114],{"href":3113},"\u002Fautomating-side-hustle-operations-with-apis\u002Fscheduling-data-pipelines-with-cron\u002Fapscheduler-vs-celery-beat\u002F","APScheduler vs Celery beat"," — picking the scheduler that fires the sweep without duplicating it across replicas.",[3117,3118,3119],"style",{},"html pre.shiki code .szBVR, html code.shiki .szBVR{--shiki-default:#D73A49;--shiki-dark:#F97583}html pre.shiki code .sVt8B, html code.shiki .sVt8B{--shiki-default:#24292E;--shiki-dark:#E1E4E8}html pre.shiki code .sj4cs, html code.shiki .sj4cs{--shiki-default:#005CC5;--shiki-dark:#79B8FF}html pre.shiki code .sScJk, html code.shiki .sScJk{--shiki-default:#6F42C1;--shiki-dark:#B392F0}html pre.shiki code .s4XuR, html code.shiki .s4XuR{--shiki-default:#E36209;--shiki-dark:#FFAB70}html pre.shiki code .sZZnC, html code.shiki .sZZnC{--shiki-default:#032F62;--shiki-dark:#9ECBFF}html .default .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}html.dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}",{"title":182,"searchDepth":200,"depth":200,"links":3121},[3122,3123,3124,3125,3126,3127,3128,3129,3130,3131],{"id":29,"depth":200,"text":30},{"id":935,"depth":200,"text":936},{"id":1026,"depth":200,"text":1027},{"id":1566,"depth":200,"text":1567},{"id":2334,"depth":200,"text":2335},{"id":2940,"depth":200,"text":2941},{"id":3025,"depth":200,"text":3026},{"id":3037,"depth":200,"text":3038},{"id":3044,"depth":200,"text":3045},{"id":3078,"depth":200,"text":3079},"Work out the exact cost and latency crossover between polling and webhooks, ship a hybrid reconciliation sweep, and handle third parties that offer no webhook.","md",{"pageTitle":3135,"type":3136,"datePublished":3137,"dateModified":3137},"Webhooks vs Polling: When to Switch and What It Costs","article","2026-07-23","\u002Fgetting-started-with-python-apis-for-builders\u002Funderstanding-rest-vs-graphql\u002Fwhen-to-use-webhooks-instead-of-polling",{"title":5,"description":3132},"getting-started-with-python-apis-for-builders\u002Funderstanding-rest-vs-graphql\u002Fwhen-to-use-webhooks-instead-of-polling\u002Findex","NGpIxj8pNGEN3Ovrzr7dh6zgVHh6lKmWkEIoM7l2ocI",{"@context":3143,"@type":3144,"mainEntity":3145},"https:\u002F\u002Fschema.org","FAQPage",[3146,3151,3154,3157,3160],{"@type":3147,"name":3050,"acceptedAnswer":3148},"Question",{"@type":3149,"text":3150},"Answer","Around 50 connected accounts at one-minute freshness. That is 2.16 million outbound requests a month and roughly 72 CPU-hours, which is where a hobby-tier instance starts throttling and you upgrade. Below 20 accounts the compute is genuinely free and the extra webhook infrastructure is not worth your time.",{"@type":3147,"name":3056,"acceptedAnswer":3152},{"@type":3149,"text":3153},"No. An hourly sweep across 500 accounts is 360,000 requests a month against 21.6 million for one-minute polling — a 98% reduction while still guaranteeing completeness. Move it to nightly and it drops to 15,000. The sweep is insurance you can afford.",{"@type":3147,"name":3062,"acceptedAnswer":3155},{"@type":3149,"text":3156},"Accept both the old and new secret for one overlap window, typically 24 hours, by trying each in turn during verification. Register the new endpoint secret with the provider first, confirm live traffic verifies against it, then drop the old one. Never rotate in a single deploy — in-flight retries carry the old signature.",{"@type":3147,"name":3068,"acceptedAnswer":3158},{"@type":3149,"text":3159},"Not if you shadow first. Run the receiver in log-only mode alongside the poller for a week and compare the two event streams. Customers see nothing until the sets match, and you keep the poller as a sweep, so the worst regression is a return to your current latency rather than lost data.",{"@type":3147,"name":3074,"acceptedAnswer":3161},{"@type":3149,"text":3162},"Return 202 immediately and absorb the burst in the queue, never in the request handler. Size the queue for peak and the workers for average, since events are not time-critical once persisted. If the provider supports it, subscribe to only the event types you act on — dropping unused types often cuts inbound volume by half.",1784887028359]