An Investigation of Data Requirements for the Detection of Depression from Social Media Posts

Background: Only a fraction of the produced social media data is usable in mental health assessment. So the problem of sufficient training data for deep learning approaches arises. Data sufficiency can be presented in terms of number of users or the number of posts per user. Objective: We examine the data need of machine learning and deep learning models for a practical system and let researcher choose best fitting models depending on the dataset type available with them. We perform distinct experiments to find the effect of these issues on depression classification by various approaches. Methods: We explored various machine learning and deep learning techniques on various data set versions, taken from Twitter and Reddit, with varying numbers of users and posts per user. Diagnosed and control users are taken in different ratios to assess the impact of an imbalanced dataset. Results: The results reveal that SVM achieved 68% accuracy in depression classification for 70 users each from diagnosed and control group. It decreases for 150 users from each group, but then regains performance for 350 and 550 users from each group. Whereas Naive Bayes got 64% for the same dataset fragment (1). We observed that accuracy decreases for 150 diagnosed users, but then regains performance for 350 and 550 users. However from deep learning algorithms, HAN and BiLSTM perform better, compared to other algorithms, as the imbalance ratio increases. Conclusion: We found, mainly, that classification accuracy increases with the number of users, number of posts per user and imbalance in the number of diagnosed versus control users. We also found that posts from Reddit have better accuracy compared to tweets.