tag:status.spotlightr.com,2005:/historySpotlightr Status - Incident History2024-03-29T05:23:12ZSpotlightrtag:status.spotlightr.com,2005:Incident/198166672024-01-24T10:00:00Z2024-01-24T11:18:17ZDDoS Attack<p><small>Jan <var data-var='date'>24</var>, <var data-var='time'>10:00</var> UTC</small><br><strong>Resolved</strong> - Incident has been resolved.<br /><br />The issue caused services to be down for ~45 minutes starting at 10:04 GTM.<br />Cause of the issue was a DDOS(denial of service) attack on our infrastructure.<br />Malicious actors have been removed and additional safeguards against this happening have been implemented.<br /><br />Thank you for your understanding and patience.</p>tag:status.spotlightr.com,2005:Incident/177461572023-07-03T08:51:37Z2023-07-03T08:51:37ZVideo Uploads Unavailable<p><small>Jul <var data-var='date'> 3</var>, <var data-var='time'>08:51</var> UTC</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Jul <var data-var='date'> 2</var>, <var data-var='time'>19:25</var> UTC</small><br><strong>Update</strong> - We are continuing to work on a fix for this issue.</p><p><small>Jul <var data-var='date'> 2</var>, <var data-var='time'>19:25</var> UTC</small><br><strong>Identified</strong> - The issue has been identified and a fix is being implemented.</p>tag:status.spotlightr.com,2005:Incident/177461512023-07-02T06:00:00Z2023-07-02T19:24:07ZVideo uploads Unavailable<p><small>Jul <var data-var='date'> 2</var>, <var data-var='time'>06:00</var> UTC</small><br><strong>Resolved</strong> - We are currently investigating this issue.</p>tag:status.spotlightr.com,2005:Incident/126454962022-10-28T15:55:50Z2022-10-28T15:55:50ZDDoS Attack<p><small>Oct <var data-var='date'>28</var>, <var data-var='time'>15:55</var> UTC</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Oct <var data-var='date'>28</var>, <var data-var='time'>12:02</var> UTC</small><br><strong>Monitoring</strong> - A fix has been implemented and we are monitoring the results.</p>tag:status.spotlightr.com,2005:Incident/126454842022-10-28T07:00:00Z2022-10-28T12:00:31ZInfrastructure under DDoS attack<p><small>Oct <var data-var='date'>28</var>, <var data-var='time'>07:00</var> UTC</small><br><strong>Resolved</strong> - The entire Spotlightr platform, including the video player, the main application, all the processing networks were unavailable for a period of 4 hours due to an extensive distributed denial of service attack.<br /><br />We have subsequently increased the performance of networks and implemented additional safeguards to ensure this doesn't continue to happen.</p>tag:status.spotlightr.com,2005:Incident/112687812022-09-26T11:47:54Z2022-09-26T11:47:54ZInfrastructure under DDoS attack.<p><small>Sep <var data-var='date'>26</var>, <var data-var='time'>11:47</var> UTC</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Sep <var data-var='date'>26</var>, <var data-var='time'>10:28</var> UTC</small><br><strong>Monitoring</strong> - A fix has been implemented and we are monitoring the results.</p><p><small>Sep <var data-var='date'>26</var>, <var data-var='time'>10:18</var> UTC</small><br><strong>Update</strong> - We are continuing to work on a fix for this issue.</p><p><small>Sep <var data-var='date'>26</var>, <var data-var='time'>09:21</var> UTC</small><br><strong>Identified</strong> - The issue has been identified.</p>tag:status.spotlightr.com,2005:Incident/102761572022-06-13T23:30:00Z2022-06-14T05:46:05ZNetworking and Downstream services<p><small>Jun <var data-var='date'>13</var>, <var data-var='time'>23:30</var> UTC</small><br><strong>Resolved</strong> - API and player services experienced short outages (~10 minutes) caused by the issue in our data center.<br />https://status.digitalocean.com/incidents/xkpsn9hbnnlz</p>tag:status.spotlightr.com,2005:Incident/101422582022-06-03T17:35:39Z2022-06-03T17:35:39ZDowngraded Services<p><small>Jun <var data-var='date'> 3</var>, <var data-var='time'>17:35</var> UTC</small><br><strong>Resolved</strong> - Increased latency on the application and player services may have caused slower service in the past couple of hours. Videos continued playing at regular speed, just loading the player took longer.</p>tag:status.spotlightr.com,2005:Incident/87798032021-12-07T09:30:00Z2021-12-07T12:44:32ZTemporary Data Base Outage<p><small>Dec <var data-var='date'> 7</var>, <var data-var='time'>09:30</var> UTC</small><br><strong>Resolved</strong> - Maintenance of the database clusters in the central storage facility to optimize performance caused a temporary caching issue where replaced IPs were still in the pool. Cache purge has solved the issue. Possible 10 minute downtime in affected regions.</p>tag:status.spotlightr.com,2005:Incident/86893272021-11-25T03:00:00Z2021-11-25T15:27:08ZTemporary Data Base Outage<p><small>Nov <var data-var='date'>25</var>, <var data-var='time'>03:00</var> UTC</small><br><strong>Resolved</strong> - One of our data base centers came offline for unscheduled maintenance at this time and data was unavailable in affected regions for up to 10 minutes. This has stopped videos temporarily and account access was not possible.<br /><br />Issue is now resolved and all data centers are fully operational.</p>tag:status.spotlightr.com,2005:Incident/84019472021-11-05T09:00:13Z2021-11-05T09:08:51ZOld vooplayer domain renewal<p><small>Nov <var data-var='date'> 5</var>, <var data-var='time'>09:00</var> UTC</small><br><strong>Resolved</strong> - Old vooplayer.com SSL expired and has been renewed. Customers who were still using the old domain may have experienced downtime during the renewal.</p>tag:status.spotlightr.com,2005:Incident/81764112021-10-11T08:03:01Z2021-10-11T08:03:01ZYoutube streaming servers<p><small>Oct <var data-var='date'>11</var>, <var data-var='time'>08:03</var> UTC</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Oct <var data-var='date'> 8</var>, <var data-var='time'>12:40</var> UTC</small><br><strong>Monitoring</strong> - A fix has been implemented and we are monitoring the results.</p><p><small>Oct <var data-var='date'> 8</var>, <var data-var='time'>12:31</var> UTC</small><br><strong>Update</strong> - Updating all YT request with appropriate viewer geographical data that should minimize errors as well as have unforeseen benefit of speeding up streamins.</p><p><small>Oct <var data-var='date'> 8</var>, <var data-var='time'>11:43</var> UTC</small><br><strong>Update</strong> - We are continuing to work on a fix for this issue.</p><p><small>Oct <var data-var='date'> 7</var>, <var data-var='time'>14:36</var> UTC</small><br><strong>Identified</strong> - We've identified an issue with the Youtube streaming servers that is affecting some Youtube videos linked to Spotlightr. This is only affecting some videos linked from Youtube. Videos uploaded to Spotlightr storage are not affected. <br /><br />If your linked Youtube video is affected, it will display a message that the "media could not be loaded". Or it will take up to 15 seconds for the video to play. The issue is with Youtube so we are monitoring it to see if/when the issue is resolved.</p>tag:status.spotlightr.com,2005:Incident/79539732021-09-09T10:46:34Z2021-09-09T10:46:34ZHong Kong CDN Downgraded<p><small>Sep <var data-var='date'> 9</var>, <var data-var='time'>10:46</var> UTC</small><br><strong>Resolved</strong> - Affected points of presents fully reset and functional.</p><p><small>Sep <var data-var='date'> 9</var>, <var data-var='time'>10:06</var> UTC</small><br><strong>Identified</strong> - Several points of presence in Hong Kong are being reset. CDN Cache purged. Temporarily switching them to nearest POPs.</p>