When robot traffic surpasses humans: the Internet enters the era of "non-human dominance", and the deep change behind data

In the history of Internet development, 2025 may be marked as a key turning point. According to the latest statistics released by Cloudflare, a leading global cloud security and content distribution network service provider, the HTTP request traffic generated by robots has surpassed that of human users for the first time, accounting for 57.5%, while human contributed requests only account for 42.5%. The symbolic meaning of this figure is far beyond its surface - the Internet, a network originally built for human communication and information acquisition, has now been officially dominated by non-human entities on the basis of the basic indicator of "request volume". This is not a sudden phenomenon, but a structural change that has lasted for many years. With the training of artificial intelligence models, the popularization of automation tools, and the rise of digital assistants, robots are reshaping the network ecosystem at a speed and scale that humans cannot reach.

Data fog and the real picture: What does 57.5% mean?

Cloudflare's statistical caliber is based on HTTP requests observed by its global network nodes, covering all website traffic passing through its reverse proxy service. What needs to be clear is that this data is not a complete picture of the total network traffic, but a sample of the Internet traffic that Cloudflare serves. Considering that Cloudflare covers about 20% of the websites and a large number of API traffic around the world, this sample is highly representative. The report points out that the exact time when robot traffic exceeded human traffic is difficult to trace, as the composition of traffic varies greatly across different industries and regions. However, data from multiple quarters between 2024 and 2025 shows that the proportion of robot requests continues to rise, ultimately stabilizing at the "absolute majority" level of 57.5%.

Furthermore, these robots are not simple crawlers or malicious scripts in the traditional sense. Cloudflare emphasizes that the statistical object is "AI agents" - programs that have the ability to simulate human behavior. They can represent users in browsing web pages, reading product details, comparing airline ticket prices, performing multi-step tasks such as booking hotels, crawling page content for large model training, and even acting as personal assistants for ordering, comparing prices, shopping, and customer service. This means that behind every HTTP request at present, it may be a human actively clicking, or more likely an AI silently initiating it in the background.

An exception worth noting is that if the statistical indicator is switched from "HTTP request count" to "application usage time", "streaming media playback time", or "total duration of infinite scrolling information flow", human users are still the absolute subject. This reveals the essence of robot traffic: they pursue "efficiency" in data collection, task execution, and content indexing, rather than immersive consumption. Although human time is limited, the request density is difficult to match that of robots. A running AI crawler can make thousands of requests in minutes, while a person may only complete a few page loads at the same time.

Regional Differentiation: Extreme Cases of Gibraltar, Singapore, and Iran

From the perspective of regional distribution, the difference in the ratio of robot to human traffic is even more thought-provoking. Gibraltar ranks first in the world with a robot traffic ratio of 92.1%, which means that less than 1 out of every 10 HTTP requests made in the region comes from real humans. Following closely behind are Singapore and Iran, both with a ratio of 76.4%. As an overseas territory of the United Kingdom, Gibraltar, with an area of only 6.7 square kilometers and a population of less than 35000, has become a hot spot for data centers, cryptocurrency mines and cross-border server hosting with loose Internet regulation and low tax rates. A large number of automated scripts and machine learning training clusters are deployed here, causing local traffic to be almost swallowed up by machines.

The situation in Singapore reflects its position as a digital hub in the Asia Pacific region. Singapore is the location of numerous technology companies' Asia Pacific headquarters, with dense cloud server clusters, financial algorithm trading centers, and a large number of AI enterprises. At the same time, the Singapore government's policy support for data centers has led to a massive outbreak of crawling, indexing, and training tasks. 76.4% of Iran's population contains a different explanation: the country's online environment is strictly controlled by the government, and residents use VPN extensively to access overseas websites. The encrypted traffic generated by VPN connections is indeed easily classified as "non-human" behavior by Cloudflare's detection mechanism; In addition, there is also a certain scale of automated information gathering activities (such as news aggregation and social media monitoring) within Iran, further increasing the proportion of robots.

These extreme cases illustrate that the distribution of robot traffic is not a random technological phenomenon, but is deeply tied to local digital infrastructure, regulatory policies, geopolitical environment, and business ecology. When a certain condition (such as low-cost electricity, relaxed laws, high-density servers) is prominent in a region, robot traffic will concentrate and "flood in".

Deep driving: AI training needs, automated business, and endless crawling

Why can robot traffic jump from "30%" to "over half" in just a few years? There are three core driving forces.

Firstly, the "data thirst" of big language models and generative AI. The training of models such as ChatGPT, Claude, Gemini, etc. requires massive, high-quality, and diverse text corpora. Although some data comes from publicly available datasets, more proprietary, real-time, and domain specific content needs to be directly crawled from websites. This has led to a sharp expansion in the scale of "intelligent crawlers" deployed by AI companies, which not only crawl HTML but also simulate login, scrolling, and clicking to obtain dynamically loaded content. It is estimated that OpenAI's GPTBot alone can make thousands of requests per second at its peak, far exceeding what any single human can achieve.

Secondly, the penetration of commercial automation. From price comparison websites, flight monitoring tools to social media management platforms, tens of thousands of businesses use robots to complete repetitive tasks. These robots are no longer simple "crawlers", but intelligent agents capable of running a complete workflow: they read product page prices, compare multiple suppliers, automatically submit orders, and even handle after-sales customer service conversations. For example, multiple API calls and web crawling of travel aggregation websites Kayak or Skyscanner are essentially robot traffic. The ROI of this type of commercial automation is extremely high, which makes enterprises willing to invest resources to maintain large-scale robot clusters.

Thirdly, adversarial crawlers and security games. Malicious robots, such as credential stuffing, credit card fraud, and content theft, have never disappeared, but have become more intelligent with the popularity of automated tools. In order to bypass the protection of security products such as Cloudflare, attackers constantly upgrade the robot's human-machine simulation capabilities - using real browser engines, randomizing user agents, and delaying simulated click rhythms. This, in turn, forces security companies to more accurately identify 'non-human behavior', leading to both parties simultaneously increasing the total amount of robot traffic. Cloudflare and other vendors' statistics on robot traffic already include two categories: legitimate and malicious, with the latter accounting for approximately 15% to 25%.

Structural impact: the "dark side" of the Internet and the reshaping of business logic

The traffic of robots exceeds that of humans, which has a profound impact on the Internet ecology and is spreading from the technical level to the business model.

From the perspective of website operators, changes in traffic composition directly impact server costs and content revenue. If a website suddenly receives a large number of AI crawling requests, even if it does not generate advertising revenue, it will consume bandwidth and CPU resources. Some publishers have started banning well-known crawlers such as GPTBot because their crawling does not generate traffic conversion (users directly obtain information through AI summaries and no longer click on the original text). This has given rise to a new "content authorization economy" - companies such as OpenAI and Google have reached payment agreements with News Corporation, Reddit, and others in exchange for legitimate crawling permissions. In the future, websites may charge separately for "human requests" and "robot requests", or set differentiated access rates.

From a user experience perspective, when a large number of robots access the same site simultaneously, it may cause server response to slow down and human users to experience page loading delays. Even more covertly, the fake traffic generated by robots may be misunderstood by advertising technology platforms as' high activity ', leading advertisers to pay for displays that are not viewed by anyone. Although multiple companies have claimed to use AI to combat fake traffic, the automation level of robot traffic is also increasing, forming an "arms race".

From the perspective of data governance, the abundance of robots requires careful interpretation of any analysis based on public traffic data. For example, the "webpage traffic" indicator may be significantly exaggerated due to machine crawlers, which can interfere with investment decisions or market research. For content creators, the ability of AI to capture and bypass paywalls is eroding the foundation of the paid subscription model. The European Union, the United States, and China have successively introduced data capture restriction regulations, but global coordination is still out of reach.

Future Trends: When will robot traffic reach its peak?

Cloudflare's data only reveals a snapshot of the current stage. With the continuous growth of data demand for AI agents (such as Microsoft Copilot and Google Gemini directly executing web operations), Internet of Things devices and auto drive system, the proportion of robots in HTTP requests is likely to exceed 60% or even 70% in the short term. But the growth curve will not be infinitely steep, as there are three major inhibitory factors.

One is the website's anti robot measures. Cloudflare's services such as Bot Management and Turnstile are constantly evolving, and the increase in payment thresholds for website operators will limit some low value crawlers. The second is regulatory pressure. The EU's Artificial Intelligence Act and General Data Protection Regulation (GDPR) require that crawling behavior must be transparent and respect the wishes of the website. Violators face huge fines, which may cause some companies to voluntarily shrink. The third is technological self limitation - many AI models are training close to data saturation, the marginal benefits of repeated crawling decrease, and companies may turn to synthetic or private data.

However, a more fundamental question is worth considering: if most HTTP requests come from machines, is the "original design intention" of the Internet still tenable? The inventor of the World Wide Web, Tim Berners Lee, once envisioned it as a public space for human information sharing, but today, this space is being inundated by 'silent eavesdroppers'. The future Internet architecture may need to distinguish between "human channel" and "machine channel" from the bottom layer, similar to the car lane and truck lane on the expressway. This data from Cloudflare is not only a statistic, but also a warning bell: we are standing at a watershed and need to redefine the rules of coexistence between humans and machines in the online world.

Conclusion

Robot HTTP requests surpass humans, and it is not a simple numbers game. It marks that the main users of Internet infrastructure have undergone a qualitative change: machines are no longer just tools for humans to use the network, but independent and dominant traffic sources. The extreme cases of Gibraltar, Singapore, and Iran demonstrate that regional policies and industrial layouts can greatly distort the composition of traffic. AI model training, business automation, and security game are the three core driving forces, and this trend will force a comprehensive restructuring of website operations, advertising pricing, content authorization, and global data governance rules. In the visible future, human users will still be the core consumers of immersive applications, but the "voting power" of HTTP requests is firmly held in the hands of machines. For the industry, understanding and adapting to this new pattern as early as possible is more practical than debating whether robot traffic is healthy.