In this article, we study the Poisson process of order k (PPoK) time-changed with an independent Lévy subordinator and its inverse, which we call, respectively, as TCPPoK-I and TCPPoK-II, through various distributional properties, long-range dependence and limit theorems for the PPoK and the TCPPoK-I. Further, we study the governing difference-differential equations of the TCPPoK-I for the case inverse Gaussian subordinator. Similarly, we study the distributional properties, asymptotic moments and the governing difference-differential equation of TCPPoK-II. As an application to ruin theory, we give a governing differential equation of ruin probability in insurance ruin using these processes. Finally, we present some simulated sample paths of both the processes.