This paper proposes a testing procedure to identify whether an asymmetric property of economic time series is caused by the common or idiosyncratic factors. To this end, I apply the test of skewness in the time series proposed by Bai and Ng (2005) to the common factor model. I show that the statistics have the standard null distributions when N and T go to infinity under the square root of T over N goes to zero where T is the dimension of time period and N is the dimension of cross section. A simulation shows that in finite samples size distortions and power loss are not substantial both in common and idiosyncratic components. I apply the proposed tests to U.S. macroeconomic time series and find that three common factors and ten idiosyncratic components particularly related to labor market and output are significantly skewed.