The engineering team has almost completed backfilling any affected resources and has confirmed with random sampling that the fix was effective.
If your on-forge.com subdomain is still not operational, please reach out to our Support Team who will be happy to help.
We apologise for any inconvenience caused.
May 11, 17:09 UTC
Monitoring -
A fix has been implemented for creating and updating new and existing on-forge.com subdomains.
Subdomains created during the incident, since approximately 13:00 UTC may not have been created. Our Engineering Team are currently rolling out a fix for those. If you require a working on-forge.com subdomain sooner, you can access the "Domains" tab of your site and edit or activate the subdomain again. This will ensure the records are created successfully.
If you choose to wait, we anticipate a fix being applied soon.
May 11, 15:52 UTC
Identified -
We have identified an issue with forge-supplied on-forge.com subdomains where newly created or recently updated subdomains are failing to resolve.
Our Engineering Team has been mobilised and are working on a resolution.
May 11, 15:39 UTC
Resolved -
This incident has been resolved. We continue to monitor for degraded performance due to ongoing DDoS activity.
May 3, 17:03 UTC
Monitoring -
We've finalized rollout of the fixes for missing PHP installations on fresh servers and installing additional PHP versions on existing servers.
Forge operations that require communications with ppa.launchpad.net may still be intermittent and slow to process while the DDoS is ongoing, but our fixes have improved the reliability of PHP installations and Forge operations.
We will continue to monitor and suggest reaching out to our support team if you are experiencing issues.
May 1, 23:51 UTC
Update -
We've deployed a fix for fresh servers and are actively trying to fix existing servers which may still encounter issues with installing additional PHP versions
May 1, 22:53 UTC
Identified -
Server provisioning and PHP install/upgrade actions may fail due to an upstream Ubuntu PHP repository (ppa.launchpadcontent.net) outage caused by a DDoS attack on its CDN. We are deploying a fix to switch to a mirror. Will update once rolled out.
May 1, 21:27 UTC
Investigating -
We are investigating reports of server provisioning failures. Next update at 21:10 UTC
May 1, 20:57 UTC