I've been thinking about your question, and I think I can add some more to the discussion.
In all the examples you provided, hydrogen is acting as the acid (it is accepting the electron) and the alkali elements are acting as bases. So, given that the acid for all the examples is the same (hydrogen), we need to analyze the quality of the base. A base with a very low electronegativity, weaker bond, and larger radius will be stronger. With this in mind, I would have to say that the order should still be CsH > RbH > NaH > LiH.