Upfirst Status - Incident history https://status.upfirst.ai Upfirst Mon, 27 Jul 2026 18:06:39 +0000 We’re seeing elevated latency, timeouts, and interrupted streaming across gpt 5.1 mini model and gpt 4.1 mini Type: Incident Duration: 6 days, 21 hours and 39 minutes Affected Components: OpenAI → Chat Completions API Jul 27, 20:52:18 GMT+0 - Resolved - All impacted services have now fully recovered. Type: Incident

Duration: 6 days, 21 hours and 39 minutes

Affected Components:

<p><small>Jul <var data-var='date'> 27</var>, <var data-var='time'>20:52:18</var> GMT+0</small><br><strong>Resolved</strong> - All impacted services have now fully recovered..</p> ]]> Mon, 27 Jul 2026 18:06:39 +0000 https://status.upfirst.ai/incident/cms3pfu7y02wd13ojlwt5t699 https://status.upfirst.ai/incident/cms3pfu7y02wd13ojlwt5t699 Elevated error rates Type: Incident Duration: 21 hours and 37 minutes Affected Components: OpenAI → Chat Completions API, OpenAI → Realtime API Jul 25, 11:46:39 GMT+0 - Monitoring - We have applied the mitigation and are monitoring the recovery. Jul 25, 11:57:02 GMT+0 - Resolved - All impacted services have now fully recovered. Type: Incident

Duration: 21 hours and 37 minutes

Affected Components: ,

<p><small>Jul <var data-var='date'> 25</var>, <var data-var='time'>11:46:39</var> GMT+0</small><br><strong>Monitoring</strong> - We have applied the mitigation and are monitoring the recovery..</p> <p><small>Jul <var data-var='date'> 25</var>, <var data-var='time'>11:57:02</var> GMT+0</small><br><strong>Resolved</strong> - All impacted services have now fully recovered..</p> ]]> Sat, 25 Jul 2026 11:35:24 +0000 https://status.upfirst.ai/incident/cms0b399p0cmu1mln0kgnldi7 https://status.upfirst.ai/incident/cms0b399p0cmu1mln0kgnldi7 Elevated error rates Type: Incident Duration: 4 days, 14 hours and 47 minutes Affected Components: OpenAI → Chat Completions API, OpenAI → Realtime API Jul 25, 10:02:52 GMT+0 - Monitoring - We have applied the mitigation and are monitoring the recovery. Jul 25, 11:08:36 GMT+0 - Resolved - All impacted services have now fully recovered. Type: Incident

Duration: 4 days, 14 hours and 47 minutes

Affected Components: ,

<p><small>Jul <var data-var='date'> 25</var>, <var data-var='time'>10:02:52</var> GMT+0</small><br><strong>Monitoring</strong> - We have applied the mitigation and are monitoring the recovery..</p> <p><small>Jul <var data-var='date'> 25</var>, <var data-var='time'>11:08:36</var> GMT+0</small><br><strong>Resolved</strong> - All impacted services have now fully recovered..</p> ]]> Sat, 25 Jul 2026 09:17:49 +0000 https://status.upfirst.ai/incident/cms07c8sd08mt0klnnd8n02ii https://status.upfirst.ai/incident/cms07c8sd08mt0klnnd8n02ii Elevated Error Rates Type: Incident Duration: 10 days, 21 hours and 19 minutes Affected Components: OpenAI → Chat Completions API, OpenAI → Realtime API Jul 23, 17:36:32 GMT+0 - Identified - We are continuing to work on implementing a mitigation. Jul 23, 17:53:36 GMT+0 - Identified - We have identified that users are experiencing elevated errors for the impacted services. We are working on implementing a mitigation. Jul 23, 17:58:26 GMT+0 - Identified - We are continuing to work on implementing a mitigation. Jul 23, 18:50:34 GMT+0 - Monitoring - We have applied the mitigation and are monitoring the recovery. Jul 23, 19:37:21 GMT+0 - Investigating - We are investigating the issue for the listed services. Jul 24, 10:06:05 GMT+0 - Investigating - We are investigating the issue for the listed services. Jul 23, 20:22:48 GMT+0 - Monitoring - We have applied the mitigation and are monitoring the recovery. Jul 24, 19:33:22 GMT+0 - Resolved - All impacted services have now fully recovered. Jul 23, 15:56:57 GMT+0 - Identified - We have identified that users are experiencing elevated errors for the impacted services. We are working on implementing a mitigation. Jul 23, 16:08:37 GMT+0 - Identified - We have identified that users are experiencing elevated errors for the impacted services. We are working on implementing a mitigation. Jul 23, 16:48:54 GMT+0 - Identified - We have identified that users are experiencing elevated errors for the impacted services. We are working on implementing a mitigation with the downstream infrastructure provider. Type: Incident

Duration: 10 days, 21 hours and 19 minutes

Affected Components: ,

<p><small>Jul <var data-var='date'> 23</var>, <var data-var='time'>17:36:32</var> GMT+0</small><br><strong>Identified</strong> - We are continuing to work on implementing a mitigation..</p> <p><small>Jul <var data-var='date'> 23</var>, <var data-var='time'>17:53:36</var> GMT+0</small><br><strong>Identified</strong> - We have identified that users are experiencing elevated errors for the impacted services. We are working on implementing a mitigation..</p> <p><small>Jul <var data-var='date'> 23</var>, <var data-var='time'>17:58:26</var> GMT+0</small><br><strong>Identified</strong> - We are continuing to work on implementing a mitigation..</p> <p><small>Jul <var data-var='date'> 23</var>, <var data-var='time'>18:50:34</var> GMT+0</small><br><strong>Monitoring</strong> - We have applied the mitigation and are monitoring the recovery..</p> <p><small>Jul <var data-var='date'> 23</var>, <var data-var='time'>19:37:21</var> GMT+0</small><br><strong>Investigating</strong> - We are investigating the issue for the listed services..</p> <p><small>Jul <var data-var='date'> 24</var>, <var data-var='time'>10:06:05</var> GMT+0</small><br><strong>Investigating</strong> - We are investigating the issue for the listed services..</p> <p><small>Jul <var data-var='date'> 23</var>, <var data-var='time'>20:22:48</var> GMT+0</small><br><strong>Monitoring</strong> - We have applied the mitigation and are monitoring the recovery..</p> <p><small>Jul <var data-var='date'> 24</var>, <var data-var='time'>19:33:22</var> GMT+0</small><br><strong>Resolved</strong> - All impacted services have now fully recovered..</p> <p><small>Jul <var data-var='date'> 23</var>, <var data-var='time'>15:56:57</var> GMT+0</small><br><strong>Identified</strong> - We have identified that users are experiencing elevated errors for the impacted services. We are working on implementing a mitigation..</p> <p><small>Jul <var data-var='date'> 23</var>, <var data-var='time'>16:08:37</var> GMT+0</small><br><strong>Identified</strong> - We have identified that users are experiencing elevated errors for the impacted services. We are working on implementing a mitigation..</p> <p><small>Jul <var data-var='date'> 23</var>, <var data-var='time'>16:48:54</var> GMT+0</small><br><strong>Identified</strong> - We have identified that users are experiencing elevated errors for the impacted services. We are working on implementing a mitigation with the downstream infrastructure provider..</p> ]]> Thu, 23 Jul 2026 15:36:02 +0000 https://status.upfirst.ai/incident/cmrxp52az00j00zqsl298dddp https://status.upfirst.ai/incident/cmrxp52az00j00zqsl298dddp Google SSO auth doesn't work Type: Incident Duration: 1 day, 1 hour and 21 minutes Affected Components: Web App Jun 25, 16:41:00 GMT+0 - Resolved - This incident has been resolved. Jun 25, 15:20:00 GMT+0 - Investigating - We are currently investigating this incident. Type: Incident

Duration: 1 day, 1 hour and 21 minutes

Affected Components:

<p><small>Jun <var data-var='date'> 25</var>, <var data-var='time'>16:41:00</var> GMT+0</small><br><strong>Resolved</strong> - This incident has been resolved..</p> <p><small>Jun <var data-var='date'> 25</var>, <var data-var='time'>15:20:00</var> GMT+0</small><br><strong>Investigating</strong> - We are currently investigating this incident..</p> ]]> Thu, 25 Jun 2026 15:20:00 +0000 https://status.upfirst.ai/incident/cmqudmohb042b2iny8jof7r6z https://status.upfirst.ai/incident/cmqudmohb042b2iny8jof7r6z The call list is not loading Type: Incident Duration: 1 day, 23 hours and 50 minutes Affected Components: Web App Jun 23, 13:28:00 GMT+0 - Investigating - We are currently investigating this incident. Jun 23, 15:18:00 GMT+0 - Resolved - This incident has been resolved. Jun 23, 15:00:00 GMT+0 - Identified - We are continuing to work on a fix for this incident. Type: Incident

Duration: 1 day, 23 hours and 50 minutes

Affected Components:

<p><small>Jun <var data-var='date'> 23</var>, <var data-var='time'>13:28:00</var> GMT+0</small><br><strong>Investigating</strong> - We are currently investigating this incident..</p> <p><small>Jun <var data-var='date'> 23</var>, <var data-var='time'>15:18:00</var> GMT+0</small><br><strong>Resolved</strong> - This incident has been resolved..</p> <p><small>Jun <var data-var='date'> 23</var>, <var data-var='time'>15:00:00</var> GMT+0</small><br><strong>Identified</strong> - We are continuing to work on a fix for this incident..</p> ]]> Tue, 23 Jun 2026 13:28:00 +0000 https://status.upfirst.ai/incident/cmqtin9xr00a02ipbyyfc403q https://status.upfirst.ai/incident/cmqtin9xr00a02ipbyyfc403q Small number of Eleven V3 TTS generations returning 429s on the Global Residency Server Type: Incident Duration: 17 hours Affected Components: ElevenLabs → Text to Speech API May 28, 11:53:00 GMT+0 - Identified - We recently identified an issue where a smaller number of Eleven V3 TTS generations were returning 429s on the global residency server. This affected users on lower priority tiers. The service has since been fully restored and is operating normally. May 28, 12:10:00 GMT+0 - Resolved - The incident was resolved. Type: Incident

Duration: 17 hours

Affected Components:

<p><small>May <var data-var='date'> 28</var>, <var data-var='time'>11:53:00</var> GMT+0</small><br><strong>Identified</strong> - We recently identified an issue where a smaller number of Eleven V3 TTS generations were returning 429s on the global residency server. This affected users on lower priority tiers. The service has since been fully restored and is operating normally..</p> <p><small>May <var data-var='date'> 28</var>, <var data-var='time'>12:10:00</var> GMT+0</small><br><strong>Resolved</strong> - The incident was resolved..</p> ]]> Thu, 28 May 2026 11:53:00 +0000 https://status.upfirst.ai/incident/cmr2zjdvk01ik0znrojy0iyrh https://status.upfirst.ai/incident/cmr2zjdvk01ik0znrojy0iyrh Elevated Latency and Errors on API Type: Incident Duration: 6 days, 1 hour and 53 minutes Affected Components: OpenAI → Chat Completions API May 27, 14:47:02 GMT+0 - Investigating - We are investigating the issue for the listed services. May 27, 15:21:59 GMT+0 - Investigating - We are investigating the issue for the listed services. May 27, 15:45:05 GMT+0 - Identified - We are still implementing the mitigation. May 27, 15:59:22 GMT+0 - Monitoring - We have applied the mitigation and are monitoring the recovery. May 27, 17:12:55 GMT+0 - Resolved - All impacted services have now fully recovered. May 27, 17:54:42 GMT+0 - Monitoring - We have received additional reports of elevated latency and errors after our previous recovery update. Mitigation remains in place, and we are monitoring recovery while we validate the remaining impact. May 27, 20:06:53 GMT+0 - Resolved - All impacted services have now fully recovered. Type: Incident

Duration: 6 days, 1 hour and 53 minutes

Affected Components:

<p><small>May <var data-var='date'> 27</var>, <var data-var='time'>14:47:02</var> GMT+0</small><br><strong>Investigating</strong> - We are investigating the issue for the listed services..</p> <p><small>May <var data-var='date'> 27</var>, <var data-var='time'>15:21:59</var> GMT+0</small><br><strong>Investigating</strong> - We are investigating the issue for the listed services..</p> <p><small>May <var data-var='date'> 27</var>, <var data-var='time'>15:45:05</var> GMT+0</small><br><strong>Identified</strong> - We are still implementing the mitigation..</p> <p><small>May <var data-var='date'> 27</var>, <var data-var='time'>15:59:22</var> GMT+0</small><br><strong>Monitoring</strong> - We have applied the mitigation and are monitoring the recovery..</p> <p><small>May <var data-var='date'> 27</var>, <var data-var='time'>17:12:55</var> GMT+0</small><br><strong>Resolved</strong> - All impacted services have now fully recovered..</p> <p><small>May <var data-var='date'> 27</var>, <var data-var='time'>17:54:42</var> GMT+0</small><br><strong>Monitoring</strong> - We have received additional reports of elevated latency and errors after our previous recovery update. Mitigation remains in place, and we are monitoring recovery while we validate the remaining impact..</p> <p><small>May <var data-var='date'> 27</var>, <var data-var='time'>20:06:53</var> GMT+0</small><br><strong>Resolved</strong> - All impacted services have now fully recovered..</p> ]]> Wed, 27 May 2026 14:47:02 +0000 https://status.upfirst.ai/incident/cmr2zlern01jj0znrp4dfs139 https://status.upfirst.ai/incident/cmr2zlern01jj0znrp4dfs139 Intermittent 500 Errors in Text-to-Speech (TTS) Generation Type: Incident Duration: 12 hours and 35 minutes Affected Components: ElevenLabs → Text to Speech API May 25, 09:02:00 GMT+0 - Monitoring - We are currently investigating an issue affecting TTS, where some generations are failing with a 500 error. May 25, 09:35:00 GMT+0 - Resolved - The issue affecting Text-to-Speech (TTS) generation, which was causing intermittent 500 errors, has been resolved. The service is now operating normally. Type: Incident

Duration: 12 hours and 35 minutes

Affected Components:

<p><small>May <var data-var='date'> 25</var>, <var data-var='time'>09:02:00</var> GMT+0</small><br><strong>Monitoring</strong> - We are currently investigating an issue affecting TTS, where some generations are failing with a 500 error..</p> <p><small>May <var data-var='date'> 25</var>, <var data-var='time'>09:35:00</var> GMT+0</small><br><strong>Resolved</strong> - The issue affecting Text-to-Speech (TTS) generation, which was causing intermittent 500 errors, has been resolved. The service is now operating normally..</p> ]]> Mon, 25 May 2026 09:47:35 +0000 https://status.upfirst.ai/incident/cmr2zjdvk01ij0znrl9sj29mr https://status.upfirst.ai/incident/cmr2zjdvk01ij0znrl9sj29mr Outbound networking issues affecting STT, TTS streaming and batch service Type: Incident Affected Components: Deepgram → Text to Speech API May 22, 21:00:01 GMT+0 - Resolved - Some API requests may have experienced elevated error rates for approximately 2-3 minutes starting at 16:43 UTC on May 22, 2026. The issue has been resolved now. Type: Incident

Affected Components:

<p><small>May <var data-var='date'> 22</var>, <var data-var='time'>21:00:01</var> GMT+0</small><br><strong>Resolved</strong> - Some API requests may have experienced elevated error rates for approximately 2-3 minutes starting at 16:43 UTC on May 22, 2026. The issue has been resolved now..</p> ]]> Fri, 22 May 2026 21:00:01 +0000 https://status.upfirst.ai/incident/cmqtj0i1600jq2knziinhwsu7 https://status.upfirst.ai/incident/cmqtj0i1600jq2knziinhwsu7 API users may see increased error rates for GPT-5.4 and GPT-5.5 Type: Incident Duration: 2 days, 17 hours and 22 minutes Affected Components: OpenAI → Chat Completions API May 20, 00:37:42 GMT+0 - Resolved - All impacted services have now fully recovered. May 19, 23:32:19 GMT+0 - Identified - We have identified that users are experiencing elevated errors for the impacted services. We are working on implementing a mitigation. May 19, 23:38:09 GMT+0 - Monitoring - We have applied the mitigation and are monitoring the recovery. May 19, 23:59:03 GMT+0 - Monitoring - We have applied the mitigation and are monitoring the recovery. Type: Incident

Duration: 2 days, 17 hours and 22 minutes

Affected Components:

<p><small>May <var data-var='date'> 20</var>, <var data-var='time'>00:37:42</var> GMT+0</small><br><strong>Resolved</strong> - All impacted services have now fully recovered..</p> <p><small>May <var data-var='date'> 19</var>, <var data-var='time'>23:32:19</var> GMT+0</small><br><strong>Identified</strong> - We have identified that users are experiencing elevated errors for the impacted services. We are working on implementing a mitigation..</p> <p><small>May <var data-var='date'> 19</var>, <var data-var='time'>23:38:09</var> GMT+0</small><br><strong>Monitoring</strong> - We have applied the mitigation and are monitoring the recovery..</p> <p><small>May <var data-var='date'> 19</var>, <var data-var='time'>23:59:03</var> GMT+0</small><br><strong>Monitoring</strong> - We have applied the mitigation and are monitoring the recovery..</p> ]]> Tue, 19 May 2026 23:32:19 +0000 https://status.upfirst.ai/incident/cmr2zletr01jl0znrus78agw2 https://status.upfirst.ai/incident/cmr2zletr01jl0znrus78agw2 EU Residency - Issue with the Agents platform, TTS and Scribe in the EU residency server Type: Incident Duration: 1 day, 6 hours and 14 minutes Affected Components: ElevenLabs → Text to Speech API May 15, 09:12:36 GMT+0 - Investigating - We are currently investigating an issue with the Agents platform, TTS and Scribe in the EU residency server. May 15, 09:30:13 GMT+0 - Monitoring - We have released an update, and the error rate is back to normal. We are currently monitoring the situation. May 15, 09:42:51 GMT+0 - Resolved - The issue has been resolved. We will publish a public write-up on this status page once the investigation is completed. Type: Incident

Duration: 1 day, 6 hours and 14 minutes

Affected Components:

<p><small>May <var data-var='date'> 15</var>, <var data-var='time'>09:12:36</var> GMT+0</small><br><strong>Investigating</strong> - We are currently investigating an issue with the Agents platform, TTS and Scribe in the EU residency server..</p> <p><small>May <var data-var='date'> 15</var>, <var data-var='time'>09:30:13</var> GMT+0</small><br><strong>Monitoring</strong> - We have released an update, and the error rate is back to normal. We are currently monitoring the situation..</p> <p><small>May <var data-var='date'> 15</var>, <var data-var='time'>09:42:51</var> GMT+0</small><br><strong>Resolved</strong> - The issue has been resolved. We will publish a public write-up on this status page once the investigation is completed..</p> ]]> Fri, 15 May 2026 09:12:36 +0000 https://status.upfirst.ai/incident/cmr2zjdta01ih0znrukc3us7n https://status.upfirst.ai/incident/cmr2zjdta01ih0znrukc3us7n Elevated error rates with GPT 5.5 Type: Incident Duration: 2 days, 8 hours and 35 minutes Affected Components: OpenAI → Chat Completions API May 11, 16:11:00 GMT+0 - Investigating - We are investigating the issue for the listed services. May 11, 16:35:00 GMT+0 - Identified - We have identified that users are experiencing elevated errors for the impacted services. We are working on implementing a mitigation. May 11, 17:59:00 GMT+0 - Resolved - All impacted services have now fully recovered. May 11, 16:54:00 GMT+0 - Monitoring - We have applied the mitigation and are monitoring the recovery. Type: Incident

Duration: 2 days, 8 hours and 35 minutes

Affected Components:

<p><small>May <var data-var='date'> 11</var>, <var data-var='time'>16:11:00</var> GMT+0</small><br><strong>Investigating</strong> - We are investigating the issue for the listed services..</p> <p><small>May <var data-var='date'> 11</var>, <var data-var='time'>16:35:00</var> GMT+0</small><br><strong>Identified</strong> - We have identified that users are experiencing elevated errors for the impacted services. We are working on implementing a mitigation..</p> <p><small>May <var data-var='date'> 11</var>, <var data-var='time'>17:59:00</var> GMT+0</small><br><strong>Resolved</strong> - All impacted services have now fully recovered..</p> <p><small>May <var data-var='date'> 11</var>, <var data-var='time'>16:54:00</var> GMT+0</small><br><strong>Monitoring</strong> - We have applied the mitigation and are monitoring the recovery..</p> ]]> Mon, 11 May 2026 17:02:24 +0000 https://status.upfirst.ai/incident/cmr2zleqc01jh0znrwhbhlyke https://status.upfirst.ai/incident/cmr2zleqc01jh0znrwhbhlyke Increased Error Rate for gpt-5.5 model in the API Type: Incident Duration: 4 days, 7 hours and 41 minutes Affected Components: OpenAI → Chat Completions API May 8, 12:31:00 GMT+0 - Investigating - We are investigating the issue for the listed services. May 8, 13:26:14 GMT+0 - Investigating - We are seeing partial recovery, but are still working on implementing additional mitigations. May 8, 13:42:57 GMT+0 - Identified - We implemented the mitigations and are observing reduction in error rates, but are still seeing higher than normal latency. May 8, 13:57:35 GMT+0 - Monitoring - We have applied the mitigation and are monitoring the recovery. May 8, 14:14:41 GMT+0 - Resolved - All impacted services have now fully recovered. Type: Incident

Duration: 4 days, 7 hours and 41 minutes

Affected Components:

<p><small>May <var data-var='date'> 8</var>, <var data-var='time'>12:31:00</var> GMT+0</small><br><strong>Investigating</strong> - We are investigating the issue for the listed services..</p> <p><small>May <var data-var='date'> 8</var>, <var data-var='time'>13:26:14</var> GMT+0</small><br><strong>Investigating</strong> - We are seeing partial recovery, but are still working on implementing additional mitigations..</p> <p><small>May <var data-var='date'> 8</var>, <var data-var='time'>13:42:57</var> GMT+0</small><br><strong>Identified</strong> - We implemented the mitigations and are observing reduction in error rates, but are still seeing higher than normal latency..</p> <p><small>May <var data-var='date'> 8</var>, <var data-var='time'>13:57:35</var> GMT+0</small><br><strong>Monitoring</strong> - We have applied the mitigation and are monitoring the recovery..</p> <p><small>May <var data-var='date'> 8</var>, <var data-var='time'>14:14:41</var> GMT+0</small><br><strong>Resolved</strong> - All impacted services have now fully recovered..</p> ]]> Fri, 8 May 2026 12:31:00 +0000 https://status.upfirst.ai/incident/cmr2zletr01jm0znr4lh2w4d9 https://status.upfirst.ai/incident/cmr2zletr01jm0znr4lh2w4d9 Users may experience elevated error rate for gpt-4o-mini in the API Type: Incident Duration: 1 day, 4 hours and 39 minutes Affected Components: OpenAI → Chat Completions API Apr 29, 05:12:25 GMT+0 - Identified - We have identified that users are experiencing elevated errors for the impacted services. We are working on implementing a mitigation. Apr 29, 05:19:19 GMT+0 - Monitoring - We have applied the mitigation and are monitoring the recovery. Apr 29, 05:41:04 GMT+0 - Resolved - All impacted services have now fully recovered. Type: Incident

Duration: 1 day, 4 hours and 39 minutes

Affected Components:

<p><small>Apr <var data-var='date'> 29</var>, <var data-var='time'>05:12:25</var> GMT+0</small><br><strong>Identified</strong> - We have identified that users are experiencing elevated errors for the impacted services. We are working on implementing a mitigation..</p> <p><small>Apr <var data-var='date'> 29</var>, <var data-var='time'>05:19:19</var> GMT+0</small><br><strong>Monitoring</strong> - We have applied the mitigation and are monitoring the recovery..</p> <p><small>Apr <var data-var='date'> 29</var>, <var data-var='time'>05:41:04</var> GMT+0</small><br><strong>Resolved</strong> - All impacted services have now fully recovered..</p> ]]> Wed, 29 Apr 2026 05:12:25 +0000 https://status.upfirst.ai/incident/cmr2zletq01jk0znrlpnlwge8 https://status.upfirst.ai/incident/cmr2zletq01jk0znrlpnlwge8 Elevated 401 errors for API endpoints Type: Incident Duration: 1 hour and 18 minutes Affected Components: OpenAI → Chat Completions API Apr 14, 01:26:24 GMT+0 - Monitoring - We have applied the mitigation and are monitoring the recovery. Apr 14, 08:41:42 GMT+0 - Resolved - All impacted services have now fully recovered. Apr 12, 20:40:24 GMT+0 - Investigating - We are investigating the issue for the listed services. Apr 13, 05:55:20 GMT+0 - Investigating - We are investigating the issue for the listed services. Apr 13, 17:10:06 GMT+0 - Identified - We've identified that intermittent 401 errors on some API endpoints are being caused by timeouts. We have applied mitigations that reduced impact in parts of the system and are continuing to deploy additional fixes and monitor recovery until error rates return to expected levels. Apr 12, 22:28:22 GMT+0 - Monitoring - We have applied the mitigation and are monitoring the recovery. Apr 13, 06:34:30 GMT+0 - Monitoring - We have applied the mitigation and are monitoring the recovery. Apr 13, 12:57:32 GMT+0 - Identified - We have identified that users are experiencing elevated errors for the impacted services. We are working on implementing a mitigation. Apr 13, 20:14:02 GMT+0 - Identified - We've identified that intermittent 401 errors on some API endpoints are being caused by timeouts. We have applied mitigations that reduced impact in parts of the system and are continuing to deploy additional fixes and monitor recovery until error rates return to expected levels. Apr 13, 23:26:27 GMT+0 - Identified - We've identified that intermittent 401 errors on some API endpoints are being caused by timeouts. We have applied mitigations that reduced impact in parts of the system and are continuing to deploy additional fixes and monitor recovery until error rates return to expected levels. Type: Incident

Duration: 1 hour and 18 minutes

Affected Components:

<p><small>Apr <var data-var='date'> 14</var>, <var data-var='time'>01:26:24</var> GMT+0</small><br><strong>Monitoring</strong> - We have applied the mitigation and are monitoring the recovery..</p> <p><small>Apr <var data-var='date'> 14</var>, <var data-var='time'>08:41:42</var> GMT+0</small><br><strong>Resolved</strong> - All impacted services have now fully recovered..</p> <p><small>Apr <var data-var='date'> 12</var>, <var data-var='time'>20:40:24</var> GMT+0</small><br><strong>Investigating</strong> - We are investigating the issue for the listed services..</p> <p><small>Apr <var data-var='date'> 13</var>, <var data-var='time'>05:55:20</var> GMT+0</small><br><strong>Investigating</strong> - We are investigating the issue for the listed services..</p> <p><small>Apr <var data-var='date'> 13</var>, <var data-var='time'>17:10:06</var> GMT+0</small><br><strong>Identified</strong> - We've identified that intermittent 401 errors on some API endpoints are being caused by timeouts. We have applied mitigations that reduced impact in parts of the system and are continuing to deploy additional fixes and monitor recovery until error rates return to expected levels..</p> <p><small>Apr <var data-var='date'> 12</var>, <var data-var='time'>22:28:22</var> GMT+0</small><br><strong>Monitoring</strong> - We have applied the mitigation and are monitoring the recovery..</p> <p><small>Apr <var data-var='date'> 13</var>, <var data-var='time'>06:34:30</var> GMT+0</small><br><strong>Monitoring</strong> - We have applied the mitigation and are monitoring the recovery..</p> <p><small>Apr <var data-var='date'> 13</var>, <var data-var='time'>12:57:32</var> GMT+0</small><br><strong>Identified</strong> - We have identified that users are experiencing elevated errors for the impacted services. We are working on implementing a mitigation..</p> <p><small>Apr <var data-var='date'> 13</var>, <var data-var='time'>20:14:02</var> GMT+0</small><br><strong>Identified</strong> - We've identified that intermittent 401 errors on some API endpoints are being caused by timeouts. We have applied mitigations that reduced impact in parts of the system and are continuing to deploy additional fixes and monitor recovery until error rates return to expected levels..</p> <p><small>Apr <var data-var='date'> 13</var>, <var data-var='time'>23:26:27</var> GMT+0</small><br><strong>Identified</strong> - We've identified that intermittent 401 errors on some API endpoints are being caused by timeouts. We have applied mitigations that reduced impact in parts of the system and are continuing to deploy additional fixes and monitor recovery until error rates return to expected levels..</p> ]]> Sun, 12 Apr 2026 20:40:24 +0000 https://status.upfirst.ai/incident/cmr2zleqd01ji0znro6z0j272 https://status.upfirst.ai/incident/cmr2zleqd01ji0znro6z0j272 Outbound networking issues affecting STT PreRecorded API requests with callbacks, TTS Batch (HTTP) requests with callbacks, and Voice Agent API BYO 3rd party providers (LLM and TTS) Type: Incident Duration: 4 days, 19 hours and 35 minutes Affected Components: Deepgram → Text to Speech API Apr 10, 17:27:56 GMT+0 - Identified - A networking incident was identified which resulted in failed callbacks on STT PreRecorded API requests and TTS Batch requests. The issue also affected Voice Agent API sessions with LLM and TTS providers. We have identified a fix and are monitoring traffic. The source of the issue is being investigated. Apr 10, 18:15:26 GMT+0 - Monitoring - A network infrastructure fix was implemented at 18:15 UTC and the behavior was immediately resolved. We are continuing to monitor carefully. Apr 13, 19:47:31 GMT+0 - Resolved - This incident has been resolved. Type: Incident

Duration: 4 days, 19 hours and 35 minutes

Affected Components:

<p><small>Apr <var data-var='date'> 10</var>, <var data-var='time'>17:27:56</var> GMT+0</small><br><strong>Identified</strong> - A networking incident was identified which resulted in failed callbacks on STT PreRecorded API requests and TTS Batch requests. The issue also affected Voice Agent API sessions with LLM and TTS providers. We have identified a fix and are monitoring traffic. The source of the issue is being investigated..</p> <p><small>Apr <var data-var='date'> 10</var>, <var data-var='time'>18:15:26</var> GMT+0</small><br><strong>Monitoring</strong> - A network infrastructure fix was implemented at 18:15 UTC and the behavior was immediately resolved. We are continuing to monitor carefully..</p> <p><small>Apr <var data-var='date'> 13</var>, <var data-var='time'>19:47:31</var> GMT+0</small><br><strong>Resolved</strong> - This incident has been resolved..</p> ]]> Fri, 10 Apr 2026 17:27:56 +0000 https://status.upfirst.ai/incident/cmqtj0i1400jp2knzkpq93tfg https://status.upfirst.ai/incident/cmqtj0i1400jp2knzkpq93tfg