Alibaba Cloud Account Wholesale: Cloud Firewall (Cloud Firewall) Mistake Blocking Logs of Normal Business Traffic and Release Rules
In the daily work of website operation and maintenance and SEO optimization, the most alarming emergencies for the team are the sudden paralysis of the core functions of the website, the crazy throwing of 403 or 502 errors by API interfaces, and even the instantaneous drop in the success rate of search engine spiders (such as Googlebot or Baidubot).
As an SEO website optimizer, I have dealt with similar "live" many times ".
Most of the time, the team will suspect that the server is down, the code has written a Bug, or it has suffered a DDoS attack. However, after pulling out the log and checking it carefully, it was discovered that the "behind the scenes" was actually their own defensive weapon--
Alibaba Cloud Firewall (Cloud Firewall)
.
As the first gateway for cloud traffic, Cloud Firewall has powerful IPS (intrusion prevention system) and access control capabilities. However, due to the special characteristics of some business requests (such as third-party payment callback, WebHook notification, specific API data synchronization, or non-standard crawler), it is easy to be misjudged as "malicious scanning" or "vulnerability attack" by the firewall and forcibly intercepted.
Intercepting the normal traffic will not only affect the conversion of real users, but also cause the search engine Spider to be unable to grab the page, directly lowering the website's collection volume and keyword ranking. Today, I will teach you how to pass from the perspective of actual combat.
Log troubleshooting
Locate false intercepts and develop accurate and safe
Release Rules
.
1. why Cloud Firewall's false interception is "fatal" to SEO and business "?
Before talking about the specific investigation steps, we first see the serious consequences of false interception.
Alibaba Cloud Firewall is usually deployed at the boundary of the Internet (north-south traffic) and between VPC networks (east-west traffic). In case of false interception:
Search engine spiders are rejected: if a crawl request initiated by a Googlebot or Baidubot triggers some common defense feature library and is intercepted, the search engine will think your site is unstable or even block access, thus reducing the crawl frequency and lowering the index weight.
Core business interruption: for example, the payment interface callback of the e-commerce website and the user login verification API are blocked, which directly leads to the failure of the order and the business conversion rate returns to zero.
Core Web Vitals metrics deteriorate: Intercepted resource requests cause the front end to wait for a long time until timeout, pulling down TTFB (first byte time) and LCP (maximum content rendering time).
In addition, when upgrading cloud firewall specifications, tuning rules, and configuring high-security protection during the promotion period, the operation and maintenance and senior site administrators must ensure that the basic operation and maintenance status of the account is normal. It is recommended to do a good job in advance when building and maintaining the protection system.
Alibaba Cloud account top-up
and budget planning to ensure adequate account balances. In this way, you can avoid the invalidation of custom release rules and protection modes due to cloud firewall arrears degradation or functional restrictions.
Forced reset, resulting in unexpected business interruption.
2. first step: how to use logs to accurately locate false interception?
When a business error is received or abnormal traffic is found,
Log analysis
It is your most authoritative basis. Alibaba Cloud Firewall provides the powerful Log Analysis (Logstore) function to support real-time SQL queries based on Log Service (SLS).
1. Enter the Cloud Firewall log console
Log on to the Alibaba Cloud Firewall console.
In the left-side navigation pane, choose Log Analysis> Log Query.
Select the corresponding traffic direction (usually Internet border firewall or VPC border firewall).
2. Use SQL statements to accurately locate intercepted traffic
You need to obtain the characteristic information of the intercepted request (for example, the public network IP of the customer, the affected destination IP, port, domain name, or the time point when the error was reported).
Enter the following common analytical SQL commands in the query box:
Query the latest interception record by interception action (Deny) and destination IP: SQLaction: Deny and dst_ip: "Your server public network IP"
Query the specific reason why a specific source IP (such as search engine spider or third-party callback IP) is blocked: SQLsrc_ip: "203.0.113.50" and action: disable
View the top 10 source IP addresses and blocking rule names that have been blocked by IPS threat intelligence in the last hour: SQLaction: deny | select src_ip, rule_name, count(*) as total group by src_ip, rule_name order by total desc limit 10
3. Extract key elements (prepare for configuration of release rules)
In the log query result, expand the log details (in JSON format) and record the following fields:
src_ip: the source IP address (that is, the party that initiates the request, such as the search engine IP or the third-party service IP).
dst_ip: Destination IP address (public IP or SLB IP of your server).
dst_port: The destination port (such as 80, 443, or a custom port).
rule_name / signature_id: The name or signature ID of the specific rule that triggers interception (for example, SQL injection interception, scanner feature matching, etc.).
app_name / proto: The protocol used (HTTP, HTTPS, TCP, and so on).
Step 2 of the 3.: Formulate scientific and accurate release rules (avoid more and more chaos)
After locating the feature of false interception, the next step is to configure the release rules. The golden rule of configuration rules is:
smallest
Principle of authority
. Do not configure directly for the sake of saving trouble.
0.0.0.0/0
The whole network segment is released, which is equivalent to turning off the firewall directly, leaving a back door for hackers to invade.
According to the type of false interception, there are two main ways to release the firewall:
Path A: False White List for IPS Threat Intelligence/Attack Interception"
If the log shows that the traffic is being
IPS Attack Protection Engine
(Signature/vulnerability feature library) manslaughter:
In the left-side menu of the Cloud Firewall console, go to Protection Configuration-> Threat Intelligence/Attack Protection.
Find the IPS interception mode configuration item and click Whitelist Management or Custom Features/Whitelisting Rules.
Click Add Whitelist Rule: Rule Type: Select Signature ID(Signature ID) or Rule Name. Scope: Fill in the affected destination IP (your server IP), or for a specific source IP(src_ip). Note: Clearly mark the release reason (e. g. release WeChat payment callback API false alarm or release Googlebot capture false alarm).
Approach B: Access Control Rules for Access Control Policies (ACLs)"
If the flow is in
Access Control (ACL)
Stopped by the default deny rule in the policy:
Go to Access Control-> Internet boundary (or VPC boundary).
Click the Inward tab, click Create Rule.
Fill in the configuration according to the principle of "Minimize Release:
Parameter item
Recommended Configuration Values
Policy Analysis
Access Source
Only fill in the third-party public network IP that needs to be released, and never set it to 0.0.0.0/0. If it is a search engine, you can use the official search engine address book of Alibaba Cloud ".
Purpose (Destination)
Specific Server IP/Address Book
Specifies the public IP address of the target ECS or SLB instance that is blocked by error.
Protocol Type
TCP / HTTP / HTTPS
Based on the actual port selection, HTTP/HTTPS is usually selected for Web services.
Destination Port
80,443
Only the ports that must be used to release the business.
Policy (Action)
Release (Allow)
Explicitly set to release.
Priority
1 (highest priority)
Cloud Firewall matches priority from top to bottom. Be sure to adjust the priority of the release rule to the highest (the lowest value) to ensure that it is hit before the global deny rule (Deny).
After the configuration is saved, the new rule will take effect within 1-2 minutes.
Step 3 of the 4.: Verify the release effect and verification tools
After the rule configuration is complete, real-time verification must be performed to ensure that the business has returned to normal.
1. Reproduction business test
Use the previously intercepted client IP to initiate the business again.
service requests (e. g. resend payment callbacks, trigger API calls).
Observe whether the HTTP return code reverts from 403/502 to 200 OK.
2. Second review of real-time log
Turn on Cloud Firewall again
Log Query
console, run the following SQL query:
SQL
src_ip: "Release Source IP" and dst_ip: "Destination IP"
Check the latest log record,
action
Whether the field has been changed from
deny
has become
allow
(Release).
3. SEO grab test (for search engine error interception)
If previously intercepted the search engine Spider by mistake:
Log in to the Google Search Console and use the URL inspection tool (URL Inspection) to click "Live Test" to see if the Googlebot can successfully render the page and read the resource.
Log in to Baidu's search resource platform and use the crawl diagnostic tool to verify whether the crawl status is restored to "successful".
5. SEO website optimizer summary: balance security and usability
In the cloud operation and maintenance system, "absolute security" and "extreme availability" often need to find a balance point through refined management.
Prevent problems before they occur: Regularly inspect the logs of Cloud Firewall and use SQL to analyze potential false positive trends. For mainstream search engines to capture IP addresses, Alibaba Cloud's built-in search engine whitelist mechanism is preferred to prevent Spider capture from being intercepted by mistake, resulting in a sharp drop in the website's SEO ranking.
Standardized rule management: When configuring release rules, you must bring clear Comment notes (indicating the business background and responsible person), and regularly clean up the temporary release rules that are no longer in use to prevent the security firewall from being useless.
Safeguard basic cloud resources: Any protection strategy is based on the premise that the cloud infrastructure is functioning properly. When designing the security defense architecture, renewing the cloud firewall, and deploying high-security IP, sorting out the budget and completing the Alibaba Cloud account recharge in a timely manner is the underlying cornerstone to ensure that the security policy continues to be online and the business runs smoothly.
mastered"
Real-time log query and positioning-> Extract feature-> Minimize configuration release-> Real-time verification
"With this standardized closed-loop process, you can calmly deal with the false interception of the cloud firewall, which not only escorts the business security, but also protects the SEO traffic of the website!