The contribution of this thesis is in incorporating the family of Gegenbauer autoregressive moving average (GARMA) models and a special subfamily called the autoregressive fractionally integrated moving average (ARFIMA) models into the mean functions of the count distributions, including Poisson, negative binomial, generalised Poisson and double Poisson within a generalised linear modelling framework. The model properties and model estimation methods are studied. For the financial and insurance application perspective, the models are applied to analyse 136 individual U.S. Commodity Futures Trading Commission (CFTC) time series of counts. The extended models incorporating Lee-Carter (LC) model are applied to study death count data sets from 16 different countries divided according to genders and age groups. Life expectancy, annuity pricing procedures and generalised annuity option (GAO) are further studied.