Measures to avoid the repetition of the ‘meal’ in October
“Reflection on social responsibility for lack of protection”
Ensure talent and technology for service stabilization
Promote triple redundancy by connecting three data centers
As for the service gridlock that occurred in October, Kakao has put in place measures to prevent a recurrence, including a threefold increase in service stabilization investments, such as system-wide duplication.
Goh Woo-chan, co-chair of the Recurrence Prevention Committee of the Emergency Response Committee, said at the developer conference “If Kakao Dev 2022 (Photo)” held on the 7th, “We have worked hard over the past five years in ensuring talent for service stabilization, technology development, and disaster recovery (DR) implementation beyond triple redundancy “We will invest more than triple the amount invested in the next five years.”
It will also hire the best information technology (IT) engineering experts in Korea and organize a dedicated IT engineering organization separate from the existing development organization. In addition, it was agreed to establish a Disaster Recovery Committee to prepare for large-scale failures, strengthen immediate response to large-scale failures, and conduct intensive disaster preparedness training. In the Ansan data center, which is under construction with a goal of completion in 2024, redundant infrastructure is being built for round-the-clock uninterrupted operation in three areas: power, cooling and communication.
GREP Chief Executive Officer Lee Hak-young, who chaired the subcommittee investigating the cause during the day, revealed that the main causes of the service failures were “insufficient duplication of data center and operations management tools and the lack of available resources”.
CEO Lee said, “I have tried to objectively analyze the cause of this failure from the perspective of a third party outside of Kakao.” Even the converter system was only installed in the Pangyo data center.
Insufficient duplication of operations management tools, lack of manpower and resources for disaster recovery, confusing communication channels for failure response, and the absence of a control tower in the early stages of a disaster were cited as problems.
It has made specific improvements, such as monitoring system multiplexing, setting up a data multiplexing facility, and operations management tool triplexing.
“If Kakao was originally an event to share Kakao’s services and technologies, this year we will start the event as a reflection on our social responsibilities that we have failed to fulfill,” said Nam Gung-hoon, Co-Chair of the Recurrence Committee of prevention of the Emergency Response Committee.
He went on to admit once again that he was wrong, saying, “Insufficient redundancy in Kakao didn’t prevent failure in the end.”
Kakao announced in October that while user data was being duplicated, there was large-scale service gridlock because developers’ operational tools were not being duplicated.
