The technical team is often concerned with new functionality, new architecture, new performance indicators, but the user is concerned only about one thing: whether I can access Internet during upgrades? In particular, WiFi certification systems, which are a direct end-user infrastructure, will flood the IT sector in half an hour if large grid breaks occur during upgrading. I have experienced two major versions of WiFi certification systems over the past two years, drawing on some lessons learned.
Let's get to the bottom of this.
The core of WiFi certification system upgrades is data migration. You have to migrate a lot of data: user account information, package and billing strategy, historical authentication logs, history billing records, device fingerprint databases, black and white lists. Not all data are processed with the same priority.
The first priority is the user account and current effective package strategy. If these data are lost or mismanaged during migration, user authentication will fail or wrongly withheld, which is the most serious production accident. The second priority is black and white lists and fingerprint databases, which affect the correctness of security strategies. The third priority is historical logs and history billing records, which will not affect current operations but will affect compliance audits.
We made one mistake when we first upgraded the data: to put all the data together for full migration, and the synchronization of data between old and new systems did not end for two days. Then we learned to cut down core accounts, verify passages, run operations, and then move historical data from backstage. During a historical data migration, the query interfaces of the old system are maintained, so if anyone has to check history periods, they can check both systems simultaneously.
The parallel operation of old and new systems is the only safe way to switch.
A lot of people like to do the program."One cut.": Set a time window, shut down the whole network, upgrade the system, restore service. This is the easiest way to do this, but it also carries the greatest risk——If anything happens to you after the promotion, you're out of your way.
We use the new and old WiFi certification system to run in parallel and then greyscale switch. The approach is that when a new system is deployed, the domain name of the authentication portal will be gradually removed from the old system at the DNS level. Ten percent traffic first, one hour observation, confirmation of the success rate, response time, error rate within normal range, 30 percent, 50 percent, 100 percent.
The impact of this process on the network equipment is also to be considered. The addresses of the Radius server on AP and AAC are to be changed, the AAA certification template is to be modified, and Portal redirected URLs are to be modified. These configuration changes on the side of the network equipment are equally not to be fully available. We try one floor before we expand to a building and eventually cover the entire campus.
A non-sensor experience on the user side is more worth time than a technology program.
The most scary thing about the upgrades is that users find out."It's changed.". Even if the authentication process is a change from two buttons to one, a user calls and complains"We're not getting on the net.". So the insensitive migration is to be extreme at the user level.
First, the UI of Portal's authentication page did almost exactly the same thing as the old system and the user could not see any visual changes. Second, if the user had previously checked the old system, it would have been able to use a new system."Remember Device"Or..."Auto Login", the preference is retained in the new system and cannot be reset by the user. Third, if a user fails to authenticate on the day of the upgrade because it has switched to the new system, the system returns automatically to the old system and records failure, and does not allow the user to repeat and abandon it.
There is also one easy to ignore: a user notification after an upgrade."The authentication system has been upgraded and your online experience is not affected."It's a big difference if you don't write that.——It is written that users know about changes but are told to be honest; without it, users may wonder if they have been infected or blacked.
Finally, a lesson: no matter how well you are prepared, the upgrade window will have unexpected problems. So leave yourself time and room for rollbacks. Our principle is that if any link starts with an unplanned problem within an hour of the upgrade, we'll roll back and rise again. We'd rather spend another week preparing, not hanging on the line with a cold sweat."Advance Migration". It is to run the entire migration process with a complete backup of production data in the test environment, without skipping any steps. Advance migration exposes many details that are not visible on documents: for example, inconsistencies in data types in an older and new system result in import failures, for example, certain special characters being cut off during data synchronization, for example, where the compatibility of the new system with the password Hashi algorithm is less than expected. These problems are detected in previews, and you can write script restoration before official upgrade; if it is discovered only during formal upgrades, then there will be only urgent stoppage of bleeding and then temporary reprogramming, and risk and stress are not a magnitude. There must also be a communication security plan on the day of upgrade, because when there is a problem with the WiFi certification system, users’ first response is to use mobile phone traffic to access the Internet and make complaints, and the number of calls in the service may increase dramatically. If the client group does not know that an upgrade has occurred, it will cause more confusion. So we'll have to give the customer service team advance notice of how to answer the user’s question, tell them what to do, what to do, what to do to the technical team, and what can reassure the user.