Azure Microsoft Cloud Account Purchase Daichong Talk about Global Distributed Database: How Did I Use Cosmos DB to Get Operation and Maintenance Out of "Fire Pit"
Friends who have worked in cloud native architecture and global business have probably experienced the dark moments that make people feel numb:
At 3: 00 a.m. Beijing time, the mobile phone vibrated wildly, and the monitoring system sounded the highest-level alarm-"European user access delay soared", "serious conflict in cross-regional data synchronization" and "CPU of the main database soared to 100 percent without response". At this time, you got up with dark circles under your eyes, connected to VPN, and began to face a bunch of dazzling sub-library and sub-table strategies, master-slave synchronization delay, Binlog loss, and indexes that are never adjustable.
Azure Microsoft Cloud account purchase and top-up service
To be honest, the traditional database architecture in the face of real globalization, high concurrent business, operation and maintenance costs is simply a bottomless pit.
A while ago, I had a drink with some old friends who worked on architecture, and everyone was still complaining: "When will there be a database that can synchronize data to all corners of the world without worrying about fragmentation, expansion and lock conflicts every day?" in fact, over the years, everyone has tried various schemes in the production environment, from self-built MySQL/PostgreSQL clusters to various open source distributed NoSQL, and finally found that if the words "global distribution" and "fast operation and maintenance" are to be truly implemented,
Azure Cosmos DB
It is a benchmark that cannot be bypassed.
Today, instead of talking about the obscure official white papers and dry architecture diagrams, I would like to talk to you from the perspective of a front-line technician about how Cosmos DB helps us to play out the operation and maintenance experience of global distributed NoSQL.
1. the traditional global distributed pain: why do you always work late at night?
Before talking about Cosmos DB, let's look at the pain points of the traditional scheme.
Azure Microsoft Cloud account purchase and top-up service
Let's say your business goes out to sea and users are scattered across North America, Europe and Southeast Asia. In order for local users to have a smooth access experience, you must deploy database nodes locally. At this point, trouble arises:
Data synchronization delay and conflict (Split-Brain): The physical delay of cross-border optical cables is objective. If a US user and a European user modify the same piece of data at the same time, how do you resolve the conflict? Is it overwrite by timestamp? Or write complex business compensation logic?
Expansion is comparable to "major surgery": the business suddenly broke out and the single machine could not carry it. Do sub-library and sub-table? Changing code, migrating data, and checking data consistency are all production accidents.
Black hole of operation and maintenance monitoring: if there are more nodes, the operation and maintenance surface will be broken. Which node is full of disks? Which node's network is shaking? The operation and maintenance personnel are exhausted just by looking at various monitoring dashboards every day.
What we need is not more configuration parameters,
Out-of-the-box global scale and true "no operations (Serverless/Managed)" experience
.
2. to Disassemble Azure Cosmos DB: Why Can It Do "Fast Operation and Maintenance"
"?
People who come into contact with Azure Cosmos DB for the first time are often shocked by its several core features. Combined with our actual landing experience in recent years, it has several killer features in operation and maintenance:
1. The real "where the mouse clicks, where the data goes": global multi-activity and one-click replication
In the traditional architecture, if you want to add an overseas data center, the DBA team has to plan the network, deploy the instance, and configure the replication link several weeks in advance.
On Cosmos DB's Azure Portal, things get extremely violent and elegant:
Take a fancy to the computer room of which country/region, click directly on the map with the mouse, click save, and within a few minutes, the global data will automatically complete multi-activity synchronization.
Multi-Master: It supports not only global multi-read, but also global multi-write. This means that North American users write US nodes and European users write European nodes. The bottom layer automatically uses multi-version concurrency control (MVCC) and conflict resolution policies to help you.
Operation and maintenance liberation: you don't need to write scripts to monitor the master-slave delay, and you don't need to worry about the business interruption caused by the hanging of nodes in a certain area. It comes with a high availability SLA (such as a 99.999 percent availability guarantee), and the cloud vendor has a full package of how to do disaster tolerance and data redundancy at the bottom.
2. Farewell to DBA's Nightmare: Automatic Indexing (Automatic Indexing)
Anyone who has written NoSQL knows that although the structure is flexible, if the business query changes and the index is not built, the full table scan will make the database hang up in minutes.
Azure Microsoft Cloud account purchase and top-up service
Traditional NoSQL often requires DBAs to manually analyze query statements and carefully design secondary indexes and composite indexes.
Cosmos DB does a very "brutal" but extremely friendly thing to development and operations:
By default, all attributes of all data are automatically indexed (Schema-agnostic Indexing).
Development is cool: you save a JSON document and add any new fields to it. you can directly use it for WHERE conditional query without modifying any database definition.
Operation and maintenance cool: there is no need to tune the B- Tree index at all. Although it consumes a certain amount of storage space in the background to maintain the index, for the operation and maintenance personnel, one less alarm caused by "missing the index" is to live ten years less.
3. Predictable performance at the millisecond level: fine control of RU/s (request unit)
In the cloud era, the balance between cost and performance will always be the core KPI of operation and maintenance. Cosmos DB introduced
RU/s (Request Unit)
The concept of.
Whether your query reads a simple Key-Value or performs a complex SQL union query, it can consume exactly how many RUs.
You can for each container (C
container) or even the entire database to set the throughput. Not only does it support manual adjustment, but it also supports automatic scaling (Autoscale)-RU is automatically filled during peak business hours, and RU is automatically lowered during late-night lows. Bills are never wasted.
With the serverless (Serverless) mode, for that kind of small business or test environment with irregular traffic, it is really "paid according to the actual request volume without spending any money".
3. production environment to avoid the pit guide: the trade-off behind the speed of operations.
Although Cosmos DB has achieved the ultimate in operation and maintenance experience, as a veteran who has crawled in the pit many times, I would like to remind everyone:
There is no perfect architecture—only the one that best fits the specific scenario.
Rapid operations do not mean you can completely ignore everything.
1. Choosing consistency levels is an art.
The most fundamental physical law of distributed databases is
CAP theorem
. Cosmos DB offers five different levels, from strong consistency (Strong) to eventual consistency (Eventual).
If your business is financial payments and you can't tolerate the slightest illusion, you have to choose strong consistency, but this will bring higher latency and slightly lower performance.
If your business is social dynamics, likes, and page views, don't hesitate to choose session consistency (Session) or eventual consistency.
Azure Microsoft Cloud account purchase and top-up service
Operations recommendation: Don’t blindly pursue “strong consistency.” Splitting containers according to specific business segments and placing businesses that do not require high consistency at a loose consistency level can increase your system throughput geometrically.
2. The design of the partition key determines success or failure.
In the NoSQL world, the partition key is your destiny. If the partition key is not selected well (for example, by "country" and 90% of users are in China), it will lead
Hot Spotting
: The CPU and RU of one underlying physical partition are full, while other partitions are idle.
Operation and maintenance suggestion: in the early stage of architecture design, be sure to stick to the selection of partition keys with the development team. High-frequency, high-concurrency fields (such as userId or high-base fields) are usually ideal choices to ensure that data is evenly hashed to all physical nodes around the world.
4. Written at the end
Looking back, our generation of people who do cloud computing and system operation and maintenance are very lucky.
In the past, most of our energy was spent on the operation and maintenance of the underlying machines, disk expansion, and how to manually repair these "dirty work" when the master-slave synchronization was hung up. The core value of cloud-native global distributed NoSQL databases such as Azure Cosmos DB is actually to "reduce the complexity of traditional database operation and maintenance.
It combines global multi-activity, elastic expansion, automatic indexing, and high-availability disaster tolerance, which originally required a huge DBA.
Complex technology that teams can handle is packaged into a few simple buttons and an intuitive API.
Let the database return to the data itself, so that the operation and maintenance from the cumbersome underlying failure to focus on the business architecture, this is what the cloud native era should look like. The next time you encounter a late-night alarm, perhaps you can think about it: Is it time to move those old architectures to a cloud-native ecosystem that is truly globally resilient?
Azure Microsoft Cloud account purchase and top-up service
