Join us now at the IRC channel.
| *Disconnected (Connection timed out). | Jan 24 04:40 | |
| **** ENDING LOGGING AT Thu Jan 24 04:40:05 2013 | ||
| **** BEGIN LOGGING AT Thu Jan 24 04:40:34 2013 | ||
| *Now talking on #boycottnovell | Jan 24 04:40 | |
| *Topic for #boycottnovell is: TechRights.org | Channel #boycottnovell for http://TechRights.org | Free Software Sentry – watching and reporting maneuvers of those who oppose software freedom :: please also join channels #techrights and #boycottnovell-social | Jan 24 04:40 | |
| *Topic for #boycottnovell set by schestowitz at Fri May 7 00:19:56 2010 | Jan 24 04:40 | |
| -ChanServ-[#boycottnovell] Welcome to the #boycottnovell channel | Jan 24 04:40 | |
| *ChanServ gives channel operator status to schestowitz_log_ | Jan 24 04:40 | |
| *puppywatch (~PuppyWatc@host86-180-249-44.range86-180.btcentralplus.com) has joined #boycottnovell | Jan 24 04:40 | |
| *pidgin_log (~roy@host86-180-249-44.range86-180.btcentralplus.com) has joined #boycottnovell | Jan 24 04:41 | |
| *Disconnected (Connection reset by peer). | Jan 24 04:49 | |
| **** ENDING LOGGING AT Thu Jan 24 04:49:09 2013 | ||
| **** BEGIN LOGGING AT Thu Jan 24 07:01:57 2013 | ||
| *Now talking on #boycottnovell | Jan 24 07:01 | |
| *Topic for #boycottnovell is: TechRights.org | Channel #boycottnovell for http://TechRights.org | Free Software Sentry – watching and reporting maneuvers of those who oppose software freedom :: please also join channels #techrights and #boycottnovell-social | Jan 24 07:01 | |
| *Topic for #boycottnovell set by schestowitz at Fri May 7 00:19:56 2010 | Jan 24 07:01 | |
| -ChanServ-[#boycottnovell] Welcome to the #boycottnovell channel | Jan 24 07:01 | |
| *ChanServ gives channel operator status to schestowitz_log | Jan 24 07:01 | |
| *uykhjkhj (~liberty@host86-180-249-44.range86-180.btcentralplus.com) has joined #boycottnovell | Jan 24 07:48 | |
| *Disconnected (Connection reset by peer). | Jan 24 08:48 | |
| **** ENDING LOGGING AT Thu Jan 24 08:48:47 2013 | ||
| **** BEGIN LOGGING AT Thu Jan 24 10:39:21 2013 | ||
| *Now talking on #boycottnovell | Jan 24 10:39 | |
| *Topic for #boycottnovell is: TechRights.org | Channel #boycottnovell for http://TechRights.org | Free Software Sentry – watching and reporting maneuvers of those who oppose software freedom :: please also join channels #techrights and #boycottnovell-social | Jan 24 10:39 | |
| *Topic for #boycottnovell set by schestowitz at Fri May 7 00:19:56 2010 | Jan 24 10:39 | |
| -ChanServ-[#boycottnovell] Welcome to the #boycottnovell channel | Jan 24 10:39 | |
| *ChanServ gives channel operator status to schestowitz_log | Jan 24 10:39 | |
| schestowitz | > Hi, Roy, | Jan 24 11:10 |
|---|---|---|
| schestowitz | > | Jan 24 11:10 |
| schestowitz | > [12:56] <schestowitz> iophk: thanks, the site, BTW, has been | Jan 24 11:10 |
| schestowitz | > having load issues, bing playd a role | Jan 24 11:10 |
| schestowitz | > | Jan 24 11:10 |
| schestowitz | > I've seen in the /var/log/httpd/error.log that once or twice a day the | Jan 24 11:10 |
| schestowitz | > max child processes gets exceeded. We still have RAM so that could be | Jan 24 11:10 |
| schestowitz | > boosted a little. But it's not a pressing problem like with the down | Jan 24 11:10 |
| schestowitz | > time. | Jan 24 11:10 |
| schestowitz | > | Jan 24 11:10 |
| schestowitz | > [Wed Jan 23 03:51:56 2013] [error] server reached MaxClients | Jan 24 11:10 |
| schestowitz | > setting, consider raising the MaxClients setting | Jan 24 11:10 |
| schestowitz | > [Wed Jan 23 17:50:10 2013] [error] server reached MaxClients | Jan 24 11:10 |
| schestowitz | > setting, consider raising the MaxClients setting | Jan 24 11:10 |
| schestowitz | > [Wed Jan 23 19:21:55 2013] [error] server reached MaxClients | Jan 24 11:10 |
| schestowitz | > setting, consider raising the MaxClients setting | Jan 24 11:11 |
| schestowitz | > [Wed Jan 23 19:50:47 2013] [error] server reached MaxClients | Jan 24 11:11 |
| schestowitz | > setting, consider raising the MaxClients setting | Jan 24 11:11 |
| schestowitz | > [Wed Jan 23 20:10:52 2013] [error] server reached MaxClients | Jan 24 11:11 |
| schestowitz | > setting, consider raising the MaxClients setting | Jan 24 11:11 |
| schestowitz | > [Wed Jan 23 20:11:42 2013] [error] server reached MaxClients | Jan 24 11:11 |
| schestowitz | > setting, consider raising the MaxClients setting | Jan 24 11:11 |
| schestowitz | > [Wed Jan 23 20:13:15 2013] [error] server reached MaxClients | Jan 24 11:11 |
| schestowitz | > setting, consider raising the MaxClients setting | Jan 24 11:11 |
| schestowitz | > [Wed Jan 23 20:27:01 2013] [error] server reached MaxClients | Jan 24 11:11 |
| schestowitz | > setting, consider raising the MaxClients setting | Jan 24 11:11 |
| schestowitz | > [Wed Jan 23 20:33:26 2013] [error] server reached MaxClients | Jan 24 11:11 |
| schestowitz | > setting, consider raising the MaxClients setting | Jan 24 11:11 |
| schestowitz | > [Wed Jan 23 23:27:19 2013] [error] server reached MaxClients | Jan 24 11:11 |
| schestowitz | > setting, consider raising the MaxClients setting | Jan 24 11:11 |
| schestowitz | > | Jan 24 11:11 |
| schestowitz | > I've been looking at the logs trying to guess where the down time | Jan 24 11:11 |
| schestowitz | > comes from. In some of the messages from the 5-minute script, it is | Jan 24 11:11 |
| schestowitz | > clear that the server is still running. In others it is clear that it | Jan 24 11:11 |
| schestowitz | > is not running. Also, the server seems to be unavailable at midnight | Jan 24 11:11 |
| schestowitz | > UTC. | Jan 24 11:11 |
| schestowitz | > | Jan 24 11:11 |
| schestowitz | > About the usage logs, how is the reverse proxy set up in regards to | Jan 24 11:11 |
| schestowitz | > usage statistics? Is each hit reflected in the httpd access log? | Jan 24 11:11 |
| schestowitz | I am not entirely sure, but I assume so. We never had Varnish logs, so checking original IPs, for example, is something I was never able to do. | Jan 24 11:11 |
| *pidgin_log (~roy@host86-180-249-44.range86-180.btcentralplus.com) has joined #boycottnovell | Jan 24 11:16 | |
| schestowitz | > Also in /var/log/http/error.log are the following kinds of errors, | Jan 24 11:53 |
| schestowitz | > without timestamps: | Jan 24 11:53 |
| schestowitz | > | Jan 24 11:53 |
| schestowitz | > ... | Jan 24 11:53 |
| schestowitz | > /usr/bin/diff3: No newline at end of file | Jan 24 11:53 |
| schestowitz | > /usr/bin/diff3: No newline at end of file | Jan 24 11:53 |
| schestowitz | > /usr/bin/diff3: No newline at end of file | Jan 24 11:53 |
| schestowitz | > /usr/bin/diff3: No newline at end of file | Jan 24 11:53 |
| schestowitz | > /usr/bin/diff3: standard output: Broken pipe | Jan 24 11:53 |
| schestowitz | > /usr/bin/diff3: standard output: Broken pipe | Jan 24 11:53 |
| schestowitz | > /usr/bin/diff3: No newline at end of file | Jan 24 11:53 |
| schestowitz | > /usr/bin/diff3: No newline at end of file | Jan 24 11:53 |
| schestowitz | > /usr/bin/diff3: write failed | Jan 24 11:53 |
| schestowitz | > ... | Jan 24 11:53 |
| schestowitz | > | Jan 24 11:53 |
| schestowitz | > Asking around on a forum, it was suggested to increase a memory limit | Jan 24 11:53 |
| schestowitz | > in LocalSettings.php, e.g. | Jan 24 11:53 |
| schestowitz | > | Jan 24 11:53 |
| schestowitz | > ini_set( 'memory_limit', '32M' ); | Jan 24 11:53 |
| schestowitz | > | Jan 24 11:53 |
| schestowitz | > The current line is remmed out in | Jan 24 11:53 |
| schestowitz | > /home/boycottn/public_html/wiki/LocalSettings.php. Is the one in | Jan 24 11:53 |
| schestowitz | > /etc/php.ini what is used? Should we mess with it and see if it gets | Jan 24 11:53 |
| schestowitz | > rid of the diff3 error? | Jan 24 11:53 |
| schestowitz | It's worth trying. I've been struggling so much with uptime recently. Went to sleep an hour later because of that last night. | Jan 24 11:53 |
| schestowitz | I noticed that, in cases where httpd does not start, killall httpd needs to be run. It's all rather oblique; it seems like performance, not settings, plays a role in it. | Jan 24 11:53 |
| schestowitz | > I wonder if that can be automated. Though it would be better to fix | Jan 24 12:14 |
| schestowitz | > the root of the problem. | Jan 24 12:14 |
| schestowitz | > | Jan 24 12:14 |
| schestowitz | > I'm seeing now that "Latest Stories" are not listed on the start page. | Jan 24 12:14 |
| schestowitz | > | Jan 24 12:14 |
| schestowitz | > PS. 'talk' does not seem to work on the server | Jan 24 12:14 |
| schestowitz | The server has been a total mess over the past day or so. I waste a lot of my time trying to keep page serving going. I don't want to nag Tracy over it. | Jan 24 12:14 |
| schestowitz | The whole site has been pretty well fucked for the past half an hour. I can't even get to write my posts or access it. | Jan 24 12:36 |
| schestowitz | > I'm seeing that. I get 503 errors in the browser but can't find what | Jan 24 12:43 |
| schestowitz | > is wrong in the logs. | Jan 24 12:43 |
| schestowitz | It's a mess. | Jan 24 12:43 |
| schestowitz | > | Jan 24 12:56 |
| schestowitz | > PID USER PR NI VIRT RES SHR S PU %MEM TIME+ COMMAND | Jan 24 12:56 |
| schestowitz | > | Jan 24 12:56 |
| schestowitz | > 5506 mysql 15 0 1155m 776m 5064 S 16.9 25.9 107:14.72 mysqld | Jan 24 12:56 |
| schestowitz | > | Jan 24 12:56 |
| schestowitz | > 28669 apache 16 0 293m 56m 3984 S 14.0 1.9 0:09.89 httpd | Jan 24 12:57 |
| schestowitz | > | Jan 24 12:57 |
| schestowitz | > 28701 apache 18 0 341m 48m 4396 S 11.6 1.6 0:05.96 httpd | Jan 24 12:57 |
| schestowitz | > | Jan 24 12:57 |
| schestowitz | > 28616 apache 15 0 289m 53m 4280 S 11.3 1.8 0:11.03 httpd | Jan 24 12:57 |
| schestowitz | > | Jan 24 12:57 |
| schestowitz | > 28665 apache 16 0 284m 45m 3980 S 10.6 1.5 0:09.41 httpd | Jan 24 12:57 |
| schestowitz | > | Jan 24 12:57 |
| schestowitz | > 28673 apache 16 0 283m 45m 3952 S 10.0 1.5 0:06.30 httpd | Jan 24 12:57 |
| schestowitz | > | Jan 24 12:57 |
| schestowitz | > 28620 apache 16 0 343m 51m 4420 S 9.3 1.7 0:10.07 httpd | Jan 24 12:57 |
| schestowitz | > | Jan 24 12:57 |
| schestowitz | > 28638 apache 18 0 337m 46m 4404 R 6.3 1.6 0:08.79 httpd | Jan 24 12:57 |
| schestowitz | > | Jan 24 12:57 |
| schestowitz | > 28712 apache 18 0 347m 54m 4056 S 5.3 1.8 0:07.89 httpd | Jan 24 12:57 |
| schestowitz | > | Jan 24 12:57 |
| schestowitz | > 28617 apache 16 0 347m 55m 4328 R 0.7 1.9 0:08.75 httpd | Jan 24 12:57 |
| schestowitz | > | Jan 24 12:57 |
| schestowitz | > 28692 apache 15 0 286m 50m 4340 S 0.7 1.7 0:10.75 httpd | Jan 24 12:57 |
| schestowitz | 3 GB of RAM+ swap taken up while almost no pages were served. It works better now, but I'm not sure why (since minutes ago) | Jan 24 12:57 |
| schestowitz | > I'm still getting 503's | Jan 24 13:00 |
| schestowitz | Same here. The whole thing seems defunct. I don't know why all that RAM is taken up. | Jan 24 13:00 |
| schestowitz | > The main page is up again, but the others seem not to be available. | Jan 24 13:20 |
| schestowitz | My whole morning/afternoon occupied by looking at it, but I still don't know what the problem seems to be. If I kill all httpd instances (with KILL signal), then 'only' 2 GB of RAM are used. | Jan 24 13:20 |
| schestowitz | > I'm wondering why nothing shows in the logs. | Jan 24 13:28 |
| schestowitz | > It's been like that for days. When I've run top during the week, it | Jan 24 15:48 |
| schestowitz | > has show also high CPU usage for the httpds. Like this: | Jan 24 15:48 |
| schestowitz | > | Jan 24 15:48 |
| schestowitz | > | Jan 24 15:48 |
| schestowitz | > PID USER PR NI VIRT RES SHR S PU %MEM TIME+ COMMAND | Jan 24 15:48 |
| schestowitz | > | Jan 24 15:48 |
| schestowitz | > 30302 apache 16 0 272m 38m 4164 S 28.6 1.3 0:04.59 httpd | Jan 24 15:48 |
| schestowitz | > | Jan 24 15:48 |
| schestowitz | > 5506 mysql 15 0 1155m 777m 5064 S 18.3 25.9 109:01.40 mysqld | Jan 24 15:48 |
| schestowitz | > | Jan 24 15:48 |
| schestowitz | > 30226 apache 16 0 271m 36m 4192 S 15.3 1.2 0:01.89 httpd | Jan 24 15:48 |
| schestowitz | > | Jan 24 15:48 |
| schestowitz | > 30191 apache 16 0 267m 32m 3352 S 10.3 1.1 0:03.31 httpd | Jan 24 15:48 |
| schestowitz | > | Jan 24 15:48 |
| schestowitz | > 30184 apache 16 0 269m 35m 4268 S 10.0 1.2 0:04.78 httpd | Jan 24 15:48 |
| *puppywatch has quit (Remote host closed the connection) | Jan 24 17:55 | |
| *libertybox_ has quit (Read error: Connection reset by peer) | Jan 24 17:55 | |
| *libertybox_ (~liberty@host86-180-249-44.range86-180.btcentralplus.com) has joined #boycottnovell | Jan 24 17:58 | |
| *puppywatch (~PuppyWatc@host86-180-249-44.range86-180.btcentralplus.com) has joined #boycottnovell | Jan 24 17:58 | |
| schestowitz | > Hi, Roy, | Jan 24 18:59 |
| schestowitz | > | Jan 24 18:59 |
| schestowitz | > The bing bot is hammering on the site. I've added a throttle for it: | Jan 24 18:59 |
| schestowitz | > | Jan 24 18:59 |
| schestowitz | > iptables -I INPUT -i eth0 -s 216.105.40.99 -m recent --update | Jan 24 18:59 |
| schestowitz | > --hitcount 3 --seconds 540 --name BADGUY_HTTP -j REJECT | Jan 24 18:59 |
| schestowitz | > | Jan 24 18:59 |
| schestowitz | > If it does what I think it does, it should limit the amount of new | Jan 24 18:59 |
| schestowitz | > connections. I've tested it against my 3G connection and it seems to | Jan 24 18:59 |
| schestowitz | > work. Hits that draw on the cache are, of course, ignored since they | Jan 24 18:59 |
| schestowitz | > never hit our machine. But then that doesn't matter because it's the | Jan 24 19:00 |
| schestowitz | > cache anyway. | Jan 24 19:00 |
| schestowitz | > | Jan 24 19:00 |
| schestowitz | > I've put the original rules in /tmp/rules.works, but they also seem to | Jan 24 19:00 |
| schestowitz | > be in rc.local. | Jan 24 19:00 |
| schestowitz | Thanks. The site is working well not, I think. We also got some traffic today from getting lots of links (to the wiki) from dozens of Twitter users, including Assange, his mom, and Applebaum. | Jan 24 19:00 |
| *abeNd-org (~Keith@rrcs-24-173-8-98.sw.biz.rr.com) has joined #boycottnovell | Jan 24 19:07 | |
| *abeNd-org has quit (Client Quit) | Jan 24 19:08 | |
| schestowitz | > I've added this to robots.txt, the number is arbitrary and does not | Jan 24 19:16 |
| schestowitz | > match what I put in iptables: | Jan 24 19:16 |
| schestowitz | > | Jan 24 19:16 |
| schestowitz | > User-agent: msnbot | Jan 24 19:16 |
| schestowitz | > Crawl-delay: 50 | Jan 24 19:16 |
| schestowitz | > | Jan 24 19:16 |
| schestowitz | > Based on what I've read about the bing bot, we're far from alone in | Jan 24 19:16 |
| schestowitz | > getting abused by the bot. | Jan 24 19:16 |
| *Disconnected (Connection reset by peer). | Jan 24 22:07 | |
| **** ENDING LOGGING AT Thu Jan 24 22:07:12 2013 | ||
| **** BEGIN LOGGING AT Thu Jan 24 22:09:05 2013 | ||
| *Now talking on #boycottnovell | Jan 24 22:09 | |
| *Topic for #boycottnovell is: TechRights.org | Channel #boycottnovell for http://TechRights.org | Free Software Sentry – watching and reporting maneuvers of those who oppose software freedom :: please also join channels #techrights and #boycottnovell-social | Jan 24 22:09 | |
| *Topic for #boycottnovell set by schestowitz at Fri May 7 00:19:56 2010 | Jan 24 22:09 | |
| -ChanServ-[#boycottnovell] Welcome to the #boycottnovell channel | Jan 24 22:09 | |
| *ChanServ gives channel operator status to schestowitz_log_ | Jan 24 22:09 | |
| *Disconnected (Connection reset by peer). | Jan 24 22:09 | |
| **** ENDING LOGGING AT Thu Jan 24 22:09:43 2013 | ||
| **** BEGIN LOGGING AT Thu Jan 24 22:11:55 2013 | ||
| *Now talking on #boycottnovell | Jan 24 22:11 | |
| *Topic for #boycottnovell is: TechRights.org | Channel #boycottnovell for http://TechRights.org | Free Software Sentry – watching and reporting maneuvers of those who oppose software freedom :: please also join channels #techrights and #boycottnovell-social | Jan 24 22:11 | |
| *Topic for #boycottnovell set by schestowitz at Fri May 7 00:19:56 2010 | Jan 24 22:11 | |
| -ChanServ-[#boycottnovell] Welcome to the #boycottnovell channel | Jan 24 22:11 | |
| *ChanServ gives channel operator status to schestowitz_log | Jan 24 22:11 | |
| *pidgin_log (~roy@host86-180-249-44.range86-180.btcentralplus.com) has joined #boycottnovell | Jan 24 22:12 | |
| *schestowitz_log_ has quit (Ping timeout: 252 seconds) | Jan 24 22:13 | |
| *puppywatch_ has quit (Ping timeout: 240 seconds) | Jan 24 22:13 | |
| *puppywatch__ has quit (Quit: Leaving) | Jan 24 22:20 | |
| *libertybox_ (~liberty@host86-180-249-44.range86-180.btcentralplus.com) has joined #boycottnovell | Jan 24 22:20 | |
| *puppywatch__ (~PuppyWatc@host86-180-249-44.range86-180.btcentralplus.com) has joined #boycottnovell | Jan 24 22:20 | |
| *dddd (~schestowi@host86-180-249-44.range86-180.btcentralplus.com) has joined #boycottnovell | Jan 24 22:39 | |
| *dddd has quit (Changing host) | Jan 24 22:39 | |
| *dddd (~schestowi@unaffiliated/schestowitz) has joined #boycottnovell | Jan 24 22:39 | |
| *ChanServ gives channel operator status to dddd | Jan 24 22:39 | |
| *dddd is now known as schestowitz | Jan 24 22:46 | |
Generated by irclog2html.py 2.6 by Marius Gedminas - find it at mg.pov.lt!