Recently Resolved Incident
Updated: 04-08-2010 5.30 pm SGT
Symptoms:
Customers on Cloud Email unable to send and receive emails properly. The cause of the problem for Cloud email servers was due to our mail storage server freezing up and requiring a reboot. Due to the unclean shutdown, the storage server had to perform file checks which took longer than expected, resulted in extended downtime on the email service.
Resolution:
We have also further investigated the problem with the main storage server freezing and found it to be due to an issue with file locking. In order to prevent this problem we will be disabling file locking on all affected servers during off peak hours later today. The disabling will be done progressively as each server needs to be rebooted. We will monitor the status of each server closely during this process.
We will introduce new measures to prevent such cases from happening as we already implemented a 3-tier storage system (main + standby + backup) to ensure that the email storage downtime can be minimized.
Updated: 24-08-2009 1:00pm SGT
Symptoms:
Our network experienced a loss of connectivity at around 10:15am today. Upon investigation, one of the network switch/router for our shared hosting network was partially unresponsive. As it did not completely fail, failover to the secondary backup switch/router did not occur. The problem was rectified at 10:30am when we did a reboot of the primary switch/router.
Resolution:
This is an occasional problem with network equipment which I believe most of our customers would have experienced in their own offices, requiring a reboot for network connectivity to be restored. Our entire network uses enterprise network equipment and structured such a way that there is a level of redundancy that has failover to secondary equipment. However, there are rare cases such as today where we needed to step in to perform a reboot of the network equipment.
|
