I configure the window manager for "focus follows mouse" rather than the Windows-like "click to focus." If the mouse pointer indicates where my attention is, why force an extra click? Ok, the real reason is that this is what I'm used to from SunOS and Solaris in the late '80s through the mid '90s. What I say below may not apply to the click to focus crowd--I just don't know.
Under the focus tab in Xfce 4.6.1's Window Manager Tweaks is an undocumented checkbox labeled "Honor standard ICCCM focus hint." At Xfce.org, the most recent documentation is for version 4.2, and 4.2 did not have this "feature." The ICCCM is a large document, and perhaps by investing considerable time one could figure out what ICCCM focus hints are, but I don't have that time. What I have noticed is that if I select both that and "activate focus stealing prevention," applications steal focus willy-nilly. I think most users who do a lot of typing, like, say, programmers, people who create documents, people who send e-mail, etc., will not want Xfce to honor ICCCM focus hints.
The other thing I noticed recently, being new to Xfce, is that the desktop is very, very sluggish compared to Gnome. This alone was bad enough that I considered going back to Gnome. However, last week, being forced to stay at home because of the snow, I noticed another window manager option, "Delay before window receives focus." I set this to zero, or as close to zero as it would let me, and all of a sudden the performance was fairly snappy. So the Xfce default is to give the illusion of poor performance, but eliminating that delay is a huge improvement.
I like Xfce as a window manager, but the documentation is in such bad shape that I cannot recommend it to others.