cross-posted from: https://infosec.pub/post/11194362
49.6% of all internet traffic came from bots in 2023, a 2% increase over the previous year, and the highest level Imperva has reported since it began monitoring automated traffic in 2013. For the fifth consecutive year, the proportion of web traffic associated with bad bots grew to 32% in 2023, up from 30.2% in 2022, while traffic from human users decreased to 50.4%. Automated traffic is costing organizations billions (USD) annually due to attacks … More → The post Bots dominate internet activity, account for nearly half of all traffic appeared first on Help Net Security.
My CDN bill recently went from about $5 a month to over $200. Turned out it was Tictok’s spider relentlessly scraping the same content over and over again.
It was ignoring robots.txt. In the end I just had to ban their user agent in the CDN config.