Incident History (Closed)
Service Outage
Earlier today an issue came up while a backup was being performed on our AU server. With this the server crashed in doing this backup and caused the main website and license system to go offline.
This created a ripple effect on our other servers going to the US server as well. Along with this the ripple effect also passed on to some customers running software.
Update: Monitoring
This appears to be resolved for the most part with the license system fully functional again. Anyone effected please reach out to us for any possible compensation for the outage.
Affected Services: Main Website, Documentation, Licensing System, Bug Tracker, Application Programming Interface, Weblutions-AU, Fanta Bot, Content Delivery Network, Sharf System, WebPanel, Weblutions-US
Cam API Migration
We will be working to migrate our FaxCams FiveM resource API over this weekend. There might be downtime that comes with this which could prevent the script uploading images for a short period.
Update: Update
Our migration is now in progress for our domain, this means the site 'cams.faxes.foo' might not be reachable while DNS propagates.
Full resolution will be within 12 hours.
Update: Resolved
DNS has propagated now. API is back up.
Affected Services: Security Cam API
Tracker Migration
We will be working to migrate our bug and feedback system to a new host over the next 24 hours.
Downtime may occur with the service.
Update: Resolved
The migration was completed over the past weekend and everything is now fully stable.
Affected Services: Bug Tracker
Documentation Migration
We are currently migrating our documentation to a new server to increase load times and response times.
Our documentation may go offline during this period.
Update: Resolved
The migration has now been fully completed.
Affected Services: Documentation
WebPanel Maintenance
We'll be performing some upgrades to our WebPanel service over the next few days.
This will have no impact on any customer-facing services. WebPanel is a staff panel used internally.
Update: Testing
Our panel has now resumed back into service internally.
Update: Resolved
The internal staff panel is now back and stable after a couple weeks of testing.
Affected Services: WebPanel
Bug Tracker Updates
We're updating our bug tracker to a new version, it may appear offline for this duration of time.
Update: Resolved
All upgrades have been completed and the tracker should be back online.
Affected Services: Bug Tracker
Website Maintenance
We're performing some changes to our website. Hopefully this wont take long.
Update: Updated
Our site has now been fully updated. All services are back to normal.
Affected Services: Main Website
It's Update Time
Doing software update on server. Wont be too long.
Update: AU Complete
Our AU server update has completed, we are also updating our US server at this stage as well. Updates to come for that.
Update: Resolved
US server was updated earlier and everything appears to be functioning correctly now.
Affected Services: Main Website, Licensing System, Application Programming Interface, FAXES-AU, Fanta Bot, Content Delivery Network
Main website issues
We’ve been notified of an issue on our main website which is causing it to go offline at times. We’re currently monitoring this behaviour to determine the best action to take.
This does not affect the license system or should cause customer related downtime.
Update: Issue Identified
The cause of two issues have been identified including a potential user security risk. We're working on a resolution.
Update: Resolved
We have pushed a patch which includes a fix for the earlier identified security risk. We're working further to ensure this isn't the case elsewhere on our system.
Affected Services: Main Website, Content Delivery Network
Security & Backup Maintenance
We will be undertaking maintenance to perform some security checks to ensure ongoing data security along with some system backups. No downtime is expected from any services.
Estimated start time .
Affected Services: FAXES-AU, FAXES-US, Content Delivery Network
AU Server Maintenance
We're performing some updates to our AU based server tonight. Services shouldn't be affected by this maintenance.
Starting
Update: Finished
Took a bit longer than we desired. On the plus side, no downtime!
Affected Services: FAXES-AU
FAXES-US Upgrade
Our US based server is being upgrade to have more resources for us to use in the future. The expected outage is set to only last 10-20 minutes.
Update: Resolved
All services are back online.
Affected Services: Documentation, Bug Tracker, FAXES-US
Discord Bot Outage
'Fanta Bot' is experiencing uptime issues. Currently it's going on and offline on a constant basis. Until resolved this will be the case.
Affected Services: Fanta Bot
Migration
We are doing a domain migration and will have everything moved onto a new domain weblutions.au. This maintenance doesn't have an expected time limit.
Current domains on faxes.zone will be proxied to still work normally for API requests.
This is expected to take place at
Update: Correction on time
*
Update: Solved
A few days overdue as domain transfers took longer than expected. Everything has now been moved to weblutions.com with ease. No extended downtime reported.
Affected Services: Main Website, Documentation, Licensing System, CDN, FAXES-AU, FAXES-US, Bug Tracker
Dibzer Store Upgrade
DIBZERs store is being updated and tweaked. His store will remain offline while this happens. Updates to come.
Update: Resolved
Downtime is no more, issue resolved.
Affected Services: DIBZER Modifications Website
License System Outage
Our License System is undergoing some maintenance for a short period of time. This should not effect any end-users or products.
This is scheduled to start at
Update: Ongoing
Our updates are still ongoing. However, the our license system should remain online and not effect customers during this period.
We have to conduct some more tests before marking this resolved.
Update: Resolved
Our updates are now completed and issues identified are resolved.
Affected Services: Licensing System
Service Outage
Our AU based server will go offline while we upgrade the hardware within the machine to make it a bit more beefy. As this is the host server for a lot of sites downtime should be as little as possible.
This is expected to occur at .
Update: Starting
We've been sorting our some details with or host with this upgrade.
The upgrade is now starting, updates to come...
Update: Returning
Our upgrade has completed. The license system is back online.
Our store and other related services will return shortly.
Update: Resolved
All services are back online and operational. As our systems still boot up in the background there may be longer page load times for a little while which will be monitored.
Thanks for the patience.
Affected Services: Main Website, Licensing System, CDN, FaxBot, Vanilla Minecraft Server
Emergency Maintenance
Our US server is experiencing issues affecting the physical host.
From host:
We have been alerted to an issue affecting the physical host on which your machine resides. To resolve this, the host requires emergency maintenance that must be performed immediately.
expect emergency maintenance to last between 30 - 60 minutes
Update: Resolved
The issue affecting your Linode's physical host has been resolved, and no further action is required at this time.
Affected Services: Documentation, FAXES-US
Test Related Incident
There is no active incident. This incident is a live test for monitoring and bug enhancement purposes.
Affected Services: Main Website
FAXES-US Downage
An update is scheduled for our US server, this is on the . Downtime is set to last up to an hour with our host and will only affect the documentation site with our services.
Affected Services: Documentation, FAXES-US
Documentation Update
The documentation page is being updated and will be down for a few minutes while this process takes place.
Affected Services: Documentation
License System Update
Our licensing system will be undergoing an update to a new version in the early hours on the 8th FEB 2022. Expected downtime is to be less than a minute.
Affected Services: Licensing System
Major Security Outage
There is a current security outage with out host. While we're down during this period we will be pushing the maintenance from our planned downtime to now while we wait for services to resume.
This is from our host:
Our team has identified the issue affecting the Block Storage service, as well as other issues preventing boots, in our Sydney data center. A fix is in place and is currently being validated.
Updates to come!
Update: License & Status Update
Our License system located at license.faxes.zone. Along with this our status service (this page) is now fully restored.
Update: Store (main site) & CDN update
The main site located at faxes.zone and our CDN has now been restored from the security update.
Update: Documentation update
Our US machines have been updated and are now back online. This also includes the documentation found at docs.faxes.zone.
Update: Restored
All services are now fully restored.
Affected Services: Main Website, Documentation, Licensing System, CDN, FAXES-AU
AU systems reboot
Our AU hosted services (listed below) will be offline for 5 minutes max (on date below) to complete a system reboot and update for security and server health purposes.
Expected downtime to last around 5 minutes max and is on the 7th FEB 2022 12:00 AEDT.
Please be aware that non updated products may experience downtime due to our licensing system being offline as well.
Update: Date moved
As there is a current outage, this planned security work has been moved to now.
Affected Services: Main Website, Licensing System, CDN, FAXES-AU
FaxDocs Outage
Our documentation is currently offline due to a server outage. This is being resolved by the host currently and should be back online within 30 minutes from now.
Update: Resolved
Documentation services are now restored.
Affected Services: Documentation
FAXES US server maintenance
The US server will be going offline for a restart of services along with the server itself.
Update: Reboot Update
Rebooting has completed, services will be returning.
Affected Services: Documentation
Planned Maintenance
Planned maintenance has begun on my main website. Downtime is expected to be around an hour or so. Updates to follow...
Affected Services: Main Website, CDN
License System Maintenance
The license system is being updated and may be down while this happens.
Update: Resolved:
All license system updates have been performed and tested. There should no longer be downtime.
Affected Services: Licensing System
Minor Outage
A minor service outage. From the looks of it our server provider have had a small outage as multiple servers from the same provider have responded with a timeout.
Update: Resolved:
The servers are now operating normally. We will continue watching over the next few hours.
Update: Resolved:
All services are back online. This will be monitored over the next hours.
Affected Services: Disbot Website, Documentation, Bot Guys Website, [Bot] Calculations, [Bot] Daily Chatter, [Bot] Sticke
FaxUp Designed by Weblutions