SoylentNews

datapharmer writes:

At around 9:15 UTC [17 May] Salesforce pushed a database script update that was intended to add modify all permissions to a specific internal profile used by their Pardot service. Due to a scripting error View and Modify All Objects Permission was granted to all user profiles for all organizations that ever had the Pardot product, including public facing community instances. This was of course a security nightmare for customers, especially those in the Financial and Health sectors, and an emergency change was pushed around 10:00 UTC to revoke all permissions to all profiles except for administrators. No announcement was made on their status sites due to the potential for bad actors to take advantage of the security issue that was introduced until the databases could be locked down. Further action was taken around 11:00 UTC to take down PODS completely, likely to further mitigate access risk which effectively expanded the outage to customers that never used Pardot but shared an instance with customers who did.

Salesforce is holding hourly calls, and recently admitted that the script had run both in their production PODS and also in the Passive Disaster Recovery Instances, complicating the ability to recover from the issue. There is currently no ETA for recovery, though it is still their hope that they will not have any data loss. They are beginning to bring back up instances, but only administrators will have access initially and it will require additional time before administrators will be able to modify permissions and rebuild profiles and there will be a longer wait yet before profile settings can be restored from backup.

Coverage at: Geekwire, The Register, and reddit

Original Submission