Marc's Public Blog - Linux Hacking


All | Aquariums | Arduino | Btrfs | Cars | Cats | Clubbing | Dining | Dinners | Diving | Electronics | Exercising | Flying | Hiking | Linux | Linuxha | Public | Rc | Sciencemuseums | Snow | Solar | Trips

This page has a few of my blog entries about linux, but my main linux page is here
Picture of Linus


Table of Content for linux:

More pages: October 2016 August 2016 July 2016 June 2016 February 2016 January 2016 May 2015 March 2015 January 2015 October 2014 May 2014 April 2014 March 2014 January 2014 November 2013 September 2013 May 2013 March 2013 January 2013 December 2012 August 2012 May 2012 March 2012 January 2012 December 2011 August 2011 July 2011 January 2011 October 2010 August 2010 June 2010 April 2010 March 2010 January 2010 December 2009 November 2009 September 2009 August 2009 July 2009 May 2009 January 2009 December 2008 November 2008 October 2008 January 2008 November 2007 August 2007 July 2006 January 2006 August 2005 April 2005 November 2004 March 2004 February 2004



2014/10/05 Btrfs Tips: Catch Btrfs Deadlocks
π 2014-10-05 00:00 in Btrfs, Linux

Simple cronjob to detect btrfs kernel hangs

As of 3.16, btrfs can still deadlock (it happens more frequently with older kernels).

You don't want a pile of stuck processes and having a half working server that you need to debug later. The cronjob below (a template for you to modify) will alert you when your load avarage is too high, show you which processes are blocked (could be something else than btrfs), and also show you if your swap is runnig out (btrfs has memory bugs with quotas and snapshots that can eat all your memory).

SHELL=/bin/bash

# If load average is more than MAXLA, show load average and all blocked processes # As any time show anything blocked on wait_current_trans.isra.15 (used to be a btrfs hang bug) # Also show swap if it drops below MINSWAP # We pipe into bc because shell comparison doesn't do floating point. */5 * * * * nobody MAXLA=25; MINSWAP=10; if [ $(echo "$(awk '{print $1}' < /proc/loadavg) > $MAXLA" | bc) 1 ]]; then cat /proc/loadavg; ps -eo state,pid,etime,wchan:30,args | grep W |grep -v "^[RS]" ; fi; ps -eo pid,etime,wchan:30,args | grep -q [w]ait_current_trans.isra.15; if [ $(echo "$(free | grep 'Swap' | awk '{t = $2; f = $4; print (f/t*100)}') < $MINSWAP" | bc) 1 ]]; then free; fi


More pages: October 2016 August 2016 July 2016 June 2016 February 2016 January 2016 May 2015 March 2015 January 2015 October 2014 May 2014 April 2014 March 2014 January 2014 November 2013 September 2013 May 2013 March 2013 January 2013 December 2012 August 2012 May 2012 March 2012 January 2012 December 2011 August 2011 July 2011 January 2011 October 2010 August 2010 June 2010 April 2010 March 2010 January 2010 December 2009 November 2009 September 2009 August 2009 July 2009 May 2009 January 2009 December 2008 November 2008 October 2008 January 2008 November 2007 August 2007 July 2006 January 2006 August 2005 April 2005 November 2004 March 2004 February 2004