mick2me 3,033 Posted August 28, 2016 Report Share Posted August 28, 2016 I need to trace some high bandwidth use problems which has almost used up our allocated bandwidth allowance. A high proportion appears to be by Unknown bots, though we have also doubled our page hits and visits. I will take this site down later tonight or in the morning. To keep updated go to the facebook page https://www.facebook.com/Nottstalgia/?fref=ts 2 Quote Link to post Share on other sites
mick2me 3,033 Posted August 28, 2016 Author Report Share Posted August 28, 2016 Message from our server panel The account “nottstal” with primary domain “nottstalgia.com” has reached 95% of its bandwidth limit (38.3 GB/40.04 GB). Average bandwidth used per day: 1.37 GB Projected monthly bandwidth usage: 42.4 GB At the current rate of usage: The account “nottstalgia” with primary domain “nottstalgia.com” is expected to reach its bandwidth limit on 8/29/16. The account “nottstalgia” with primary domain “nottstalgia.com” is expected to exceed its bandwidth limit by 2.36 GB. Quote Link to post Share on other sites
mick2me 3,033 Posted August 28, 2016 Author Report Share Posted August 28, 2016 Visitor and member stats Day Number of visits Pages Hits Bandwidth 01 Aug 2016 1,080 13,915 50,683 573.93 MB 02 Aug 2016 1,237 15,596 55,522 646.51 MB 03 Aug 2016 1,122 13,855 49,290 582.98 MB 04 Aug 2016 1,047 13,182 49,068 552.20 MB 05 Aug 2016 953 11,973 42,834 473.48 MB 06 Aug 2016 1,001 11,593 41,579 494.37 MB 07 Aug 2016 949 11,938 43,791 467.19 MB 08 Aug 2016 1,068 12,508 48,222 518.74 MB 09 Aug 2016 1,201 13,330 48,119 588.04 MB 10 Aug 2016 1,163 12,482 45,808 501.61 MB 11 Aug 2016 382 3,181 11,781 136.44 MB 12 Aug 2016 929 9,562 39,175 443.05 MB 13 Aug 2016 1,116 10,108 40,744 456.61 MB 14 Aug 2016 1,014 11,352 43,792 520.02 MB 15 Aug 2016 1,069 12,345 45,789 565.81 MB 16 Aug 2016 1,076 16,365 60,831 831.31 MB 17 Aug 2016 947 34,318 83,811 1.24 GB 18 Aug 2016 920 30,897 88,744 1.23 GB 19 Aug 2016 929 30,430 80,412 1.29 GB 20 Aug 2016 1,014 28,077 73,285 1.30 GB 21 Aug 2016 1,037 30,372 73,860 1.30 GB 22 Aug 2016 1,148 29,570 66,798 1.18 GB 23 Aug 2016 1,172 26,000 62,317 1.11 GB 24 Aug 2016 1,102 24,636 55,964 1.06 GB 25 Aug 2016 1,128 25,550 62,125 1.14 GB 26 Aug 2016 1,020 26,199 60,011 1.12 GB 27 Aug 2016 1,223 22,472 60,815 1.20 GB 28 Aug 2016 512 7,536 17,637 407.88 MB 29 Aug 2016 0 0 0 0 30 Aug 2016 0 0 0 0 31 Aug 2016 0 0 0 0 Average 1,019 18,190 53,671 794.15 MB Total 28,559 509,342 1,502,807 21.72 GB Quote Link to post Share on other sites
mick2me 3,033 Posted August 28, 2016 Author Report Share Posted August 28, 2016 Bot activity (not included in above figures) Robots/Spiders visitors (Top 25) - Full list - Last visit 21 different robots* Hits Bandwidth Last visit Googlebot 390,331+85 5.89 GB 28 Aug 2016 - 12:05 Unknown robot (identified by 'bot' followed by a space or one of the following characters _+:,.;/\-) 148,409+5031 3.56 GB 28 Aug 2016 - 12:03 Yahoo Slurp 25,100+121 325.20 MB 28 Aug 2016 - 12:04 Unknown robot (identified by 'robot') 8,682+6354 126.09 MB 28 Aug 2016 - 11:06 Unknown robot (identified by 'crawl') 13,649+970 216.47 MB 28 Aug 2016 - 02:59 Google Web Preview 5,899 119.23 MB 28 Aug 2016 - 11:48 Googlebot-Image 3,361 16.96 MB 28 Aug 2016 - 12:00 MSNBot-media 2,051 33.97 MB 28 Aug 2016 - 12:04 Bing Preview bot 1,909 51.45 MB 28 Aug 2016 - 12:01 A PHP script 1,313 42.09 MB 28 Aug 2016 - 11:28 Unknown robot (identified by 'spider') 921+58 77.36 MB 28 Aug 2016 - 10:47 Unknown robot (identified by empty user agent string) 675+8 113.39 MB 28 Aug 2016 - 12:01 Unknown robot (identified by 'scanner') 521+121 6.07 MB 20 Aug 2016 - 04:53 Unknown robot (identified by hit on 'robots.txt') 0+550 5.16 MB 28 Aug 2016 - 11:02 Unknown robot (identified by 'bot' preceded by a space or one of the following characters _+:,.;/\-) 260 1.05 MB 26 Aug 2016 - 06:03 Unknown robot (identified by 'discovery') 228 1.62 MB 15 Aug 2016 - 16:21 Alexa (IA Archiver) 117+38 8.47 MB 27 Aug 2016 - 17:50 MSNBot 21 283.52 KB 26 Aug 2016 - 15:27 Unknown robot (identified by 'checker') 13 124.07 KB 27 Aug 2016 - 05:42 Common *nix tool for automating web document retireval. Most likely a bot. 8 204.68 KB 24 Aug 2016 - 20:58 Netcraft 1 204.42 KB 10 Aug 2016 - 08:11 * Robots shown here gave hits or traffic "not viewed" by visitors, so they are not included in other charts. Numbers after + are successful hits on "robots.txt" files. Quote Link to post Share on other sites
mick2me 3,033 Posted August 28, 2016 Author Report Share Posted August 28, 2016 25% of our bandwidth allocation is taken by bots Quote Link to post Share on other sites
carni 10,094 Posted August 28, 2016 Report Share Posted August 28, 2016 Mick, I won't pretend to understand what the problem is, but once again we thank you for the time you take to keep Nottstalgia going for us. I hope you can manage to sort things out. Quote Link to post Share on other sites
Enigma. 1,533 Posted August 28, 2016 Report Share Posted August 28, 2016 hope you get it sorted ok mick - keep up the good work Quote Link to post Share on other sites
mick2me 3,033 Posted August 28, 2016 Author Report Share Posted August 28, 2016 I just put the board offline overnight. Checked the server panel, and our host has added another 10GB of bandwidth! So the site will remain online, whilst I monitor and chase the bots! Main Domain nottstalgia.com Home Directory nottstalgia.com Last Login From CPU Usage 12 / 100 Entry Processes 3 / 20 Physical Memory Usage 43.57 / 800 I/O Usage 0 / 1524 KB/s IOPS 0 / 1024 Number of Processes 2 / 100 Disk Space Usage 3.49 GB / 9.77 GB Monthly Bandwidth Transfer 38.69 GB / 49.8 GB Email Accounts 4 / 999 Mailing Lists 0 / ∞ Addon Domains 3 / 10 Subdomains 3 / 10 Parked Domains 0 / 10 Quote Link to post Share on other sites
Cliff Ton 10,466 Posted August 29, 2016 Report Share Posted August 29, 2016 10 hours ago, mick2me said: So the site will remain online, whilst I monitor and chase the bots. Bot chaser. That's a job which careers advisors never recommended to anyone. 5 Quote Link to post Share on other sites
mick2me 3,033 Posted August 29, 2016 Author Report Share Posted August 29, 2016 True @Cliff Ton THANK YOU eriahosting.com/ The account “nottstalgia” with primary domain “nottstalgia.com” has reached 80% of its bandwidth limit (40.14 GB/49.8 GB). Average bandwidth used per day: 1.38 GB Projected monthly bandwidth usage: 42.91 GB At the current rate of usage: The account “nottstalgia” with primary domain “nottstalgia.com” is not expected to exceed their bandwidth limit. Quote Link to post Share on other sites
Gibbo 04 188 Posted August 29, 2016 Report Share Posted August 29, 2016 Eh....? Thank goodness there are those who know what they're doing!! 1 Quote Link to post Share on other sites
NewBasfordlad 3,599 Posted August 29, 2016 Report Share Posted August 29, 2016 ^^^^^^^^My thoughts exactly, I don't even speak the language never mind anything else. Quote Link to post Share on other sites
mick2me 3,033 Posted August 31, 2016 Author Report Share Posted August 31, 2016 Even wit an extra 10GB making it 50GB per month we are now cutting it fine! Googlebot hammered us for 10.32GB so far this month! An email just received from the server! The account “nottstalgia” with primary domain “nottstalgia.com” has reached 95% of its bandwidth limit (47.36 GB/49.8 GB). Average bandwidth used per day: 1.53 GB Projected monthly bandwidth usage: 47.36 GB At the current rate of usage: The account “nottstalgia” with primary domain “nottstalgia.com” is not expected to exceed their bandwidth limit. Quote Link to post Share on other sites
Gibbo 04 188 Posted August 31, 2016 Report Share Posted August 31, 2016 Nah....I still aint got a clue! Quote Link to post Share on other sites
catfan 14,793 Posted August 31, 2016 Report Share Posted August 31, 2016 Would you please hang on before closing the site down temporaly until I'm fast asleep in bed ? Quote Link to post Share on other sites
Guest Posted August 31, 2016 Report Share Posted August 31, 2016 And me!! Quote Link to post Share on other sites
mick2me 3,033 Posted September 9, 2016 Author Report Share Posted September 9, 2016 [nottstalgia.com] The account “nottstal” with primary domain “nottstalgia.com” is about to exceed its bandwidth limit (32.36 GB/40.04 GB) C cPanel for nottstal on nottstalgia.com to me 13 hours ago Details The account “nottstal” with primary domain “nottstalgia.com” has reached 80% of its bandwidth limit (32.36 GB/40.04 GB). Average bandwidth used per day: 3.6 GB Projected monthly bandwidth usage: 107.87 GB At the current rate of usage: The account “nottstal” with primary domain “nottstalgia.com” is expected to reach its bandwidth limit on 9/11/16. The account “nottstal” with primary domain “nottstalgia.com” is expected to exceed its bandwidth limit by 67.83 GB. The system generated this notice on Friday, September 9, 2016 at 1:13:55 AM UTC. Quote Link to post Share on other sites
mick2me 3,033 Posted September 9, 2016 Author Report Share Posted September 9, 2016 I am currently working on a mobile till tomorrow, unabele to get proper server access to check bandwidth usage. Expect the site to go down in the next 24-48 hours. Quote Link to post Share on other sites
mick2me 3,033 Posted September 9, 2016 Author Report Share Posted September 9, 2016 80% usage in 8 days is more than 5 time that of a couple of months ago. Google gobbled 11gb of our bandwidth last month. Quote Link to post Share on other sites
FLY2 10,108 Posted September 9, 2016 Report Share Posted September 9, 2016 It's all Chinese to me and over my head, but I got the gist of it. Quote Link to post Share on other sites
mick2me 3,033 Posted September 9, 2016 Author Report Share Posted September 9, 2016 49gb total last month, 107gb predicted this month! Quote Link to post Share on other sites
loppylugs 8,429 Posted September 9, 2016 Report Share Posted September 9, 2016 That's an awful lot of gb. Considering the nature of the site, mostly text, all photos stored offsite on a photo website, minimum use of quotes. Sounds like Google need to cut back. Is the new host using some different type of measuring criteria. Although the layout is different it shouldn't make that much difference to the amount of posting going on? Good luck with it Mick. It's enough to give you grey hair. Quote Link to post Share on other sites
Jill Sparrow 10,307 Posted September 9, 2016 Report Share Posted September 9, 2016 My hair is already grey and I haven't the foggiest clue what you're talking about! 4 Quote Link to post Share on other sites
mick2me 3,033 Posted September 9, 2016 Author Report Share Posted September 9, 2016 I cannot get into the panel till Tomorrow to find out. However you can now see the reason we dont allow lengthy quotes. Also lengthy post that are not matters subject of this boards main purpose. Quote Link to post Share on other sites
loppylugs 8,429 Posted September 9, 2016 Report Share Posted September 9, 2016 #23 Jill. C'mon Jill. I reckon you are more computer literate than that. A gigabyte is 1024 megabytes. I think? Either way thats a lot of space when you think about it. The average post probably only equals a Kbyte or two at most. So summat's using a lot of space. I wouldn't have thought all our posts for a month would come even close to 40 gig. 1 Quote Link to post Share on other sites
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.