Google Says:
It's origin is related to algebra, where in a simple formula, if there is a single unknown, it usually gets designated as an x.
"The X factor" therefore becomes "the one thing we do not conclusively know", but has lately been used to mean "an indescribable quality.
:bandid: