Description:
On January 6th at approximately 4:10 PM ET an update aimed at improving security caused a temporary disruption, affecting some user logins for approximately 6 minutes. The issue was linked to unexpected behavior in the system's connection to our Redis database, crucial for authentication.
Root Cause:
The update unintentionally allowed operations in production environments that were unnecessary, combined with temporary Redis connection issues, which impacted some user logins.
Resolution:
We reverted the changes to restore service stability.
Next Steps:
We are implementing additional safeguards and better error handling to prevent similar issues from arising in the future. We will then re-deploy the security change after the additional tests have been complete.