The main goal of this article is to construct "arithmetic Okounkov bodies" for an arbitrary pseudo-effective (1,1)-class $\alpha$ on a Kähler manifold. Firstly, using Boucksom's divisorial Zariski decompositions for pseudo-effective (1,1)-classes on compact Kähler manifolds, we prove the differentiability of volumes of big classes for Kähler manifolds on which modified nef cones and nef cones coincide; this includes Kähler surfaces. We then apply our differentiability results to prove Demailly's transcendental Morse inequality for these particular classes of Kähler manifolds. In the second part, we construct the convex body $\Delta(\alpha)$ for any big class $\alpha$ with respect to a fixed flag by using positive currents, and prove that this newly defined convex body coincides with the Okounkov body when $\alpha\in {\rm NS}_{\mathbb{R}}(X)$; such convex sets $\Delta(\alpha)$ will be called generalized Okounkov bodies. As an application we prove that any rational point in the interior of Okounkov bodies is "valuative". Next we give a complete characterisation of generalized Okounkov bodies on surfaces, and show that the generalized Okounkov bodies behave very similarly to original Okounkov bodies. By the differentiability formula, we can relate the standard Euclidean volume of $\Delta(\alpha)$ in $\mathbb{R}^2$ to the volume of a big class $\alpha$, as defined by Boucksom; this solves a problem raised by Lazarsfeld in the case of surfaces. Finally, we study the behavior of the generalized Okounkov bodies on the boundary of the big cone, which are characterized by numerical dimension.