Difference between revisions of "Supervised Machine Learning for Fake News Detection"

From Sinfronteras
Jump to: navigation, search
(→‎Summary of Results)
(→‎Summary of Results)
Line 153: Line 153:
 
|
 
|
 
|
 
|
−
|}
 
−
 
−
 
−
<br />
 
−
 
−
 
−
 
−
 
−
{| class="wikitable"
 
−
! rowspan="4" |[[Establishing an authenticity of sport news by Machine Learning Models#Algorithms|Algorithms]]
 
−
! rowspan="4" |Author
 
−
! rowspan="4" |Package
 
−
! rowspan="4" |Keyword
 
−
! colspan="8" |Accuracy
 
−
|-
 
−
! colspan="2" |[[Establishing an authenticity of sport news by Machine Learning Models#Description of the Kaggle fake news dataset|Kaggle fake news dataset]]
 
−
20,800 rows
 
−
! colspan="2" |[[Establishing an authenticity of sport news by Machine Learning Models#Description of the Fake news Detector dataset|Fake news Detector dataset]]
 
−
10,000 rows
 
−
! colspan="4" |Gofaaas Fake News Dataset
 
−
500 rows
 
−
|-
 
−
! rowspan="2" |Test data (70% of the dataset)
 
−
! rowspan="2" |Cross validation
 
−
! rowspan="2" |Test data (70% of the dataset)
 
−
! rowspan="2" |Cross validation
 
−
! rowspan="2" |Test data (70% of the dataset)
 
−
! rowspan="2" |Cross validation
 
−
! rowspan="2" |Using the Kaggle Model^
 
−
! rowspan="2" |Using the Detector Model^^
 
−
|-
 
−
|[[Establishing an authenticity of sport news by Machine Learning Models#Extreme Gradient Boosting|Naive Bayes]]
 
−
|[[wikipedia:Thomas_Bayes|Bayes, Thomas]]
 
−
|[https://cran.r-project.org/web/packages/e1071/index.html e1071]
 
−
|NB*
 
−
|[[Establishing an authenticity of sport news by Machine Learning Models#fnd_naivebayes|<math>{\color{blue}76.41%}</math>]]
 
−
|
 
−
|
 
−
|
 
−
|
 
−
|
 
−
|
 
−
|
 
−
|-
 
−
|[[Establishing an authenticity of sport news by Machine Learning Models#Support vector machine|Support vector machine]]
 
−
|Meyer et al., 2012
 
−
|We used [[Establishing an authenticity of sport news by Machine Learning Models#The RTextTools package|RTextTools]], which depends on [https://cran.r-project.org/web/packages/e1071/index.html e1071]
 
−
|SVM*
 
−
|[[Establishing an authenticity of sport news by Machine Learning Models#fnd_svm|<math>{\color{blue}95.42%}</math>]]
 
−
|
 
−
|
 
−
|
 
−
|
 
−
|
 
−
|
 
−
|
 
−
|-
 
−
|[[Establishing an authenticity of sport news by Machine Learning Models#Random forest|Random forest]]
 
−
|Liawand Wiener, 2002
 
−
|[https://cran.r-project.org/web/packages/randomForest/index.html randomForest]
 
−
|RF
 
−
|
 
−
|
 
−
|
 
−
|
 
−
|
 
−
|
 
−
|
 
−
|
 
−
|-
 
−
|[[Establishing an authenticity of sport news by Machine Learning Models#Extreme Gradient Boosting|Extreme Gradient Boosting]]
 
−
|Chen & Guestrin, 2016
 
−
|[[Establishing an authenticity of sport news by Machine Learning Models#The XGBoost package|xgboost]]
 
−
|XGBOOST*
 
−
|[[Establishing an authenticity of sport news by Machine Learning Models#fnd_xgboost|<math>{\color{red}97.42%}</math>]]
 
−
|
 
−
|
 
−
|
 
−
|
 
−
|
 
−
|
 
−
|
 
−
|-
 
−
|[[Establishing an authenticity of sport news by Machine Learning Models#General linearized models|General linearized models]]
 
−
|Friedman et al., 2010
 
−
|We used [[Establishing an authenticity of sport news by Machine Learning Models#The RTextTools package|RTextTools]], which depends on [https://cran.r-project.org/web/packages/glmnet/index.html wglmnet]
 
−
|GLMNET*
 
−
|[[Establishing an authenticity of sport news by Machine Learning Models#fnd_glmnet|<math>{\color{blue}94.58%}</math>]]
 
−
|
 
−
|
 
−
|
 
−
|
 
−
|
 
−
|
 
−
|
 
−
|-
 
−
|[[Establishing an authenticity of sport news by Machine Learning Models#Maximum entropy|Maximum entropy]]
 
−
|Jurka, 2012
 
−
|We used [[Establishing an authenticity of sport news by Machine Learning Models#The RTextTools package|RTextTools]], which depends on [https://cran.r-project.org/web/packages/maxent/index.html maxent]
 
−
|MAXENT*
 
−
|[[Establishing an authenticity of sport news by Machine Learning Models#fnd_maxent|<math>{\color{blue}96.09%}</math>]]
 
−
|
 
−
|
 
−
|
 
−
|
 
−
|
 
−
|
 
−
|
 
−
|-
 
−
| colspan="12" |* Low-memory algorithm
 
−
<nowiki>**</nowiki> Very high-memory algorithm
 
−
 
−
^
 
−
 
−
^^
 
 
|}
 
|}
  

Revision as of 19:53, 27 April 2019

Declaration


Acknowledgement

Thanks for Muhammad, Graham and Mark


Abstract


Introduction


Chapter 1


Chapter 2 - Training a Supervised Machine Learning Model for fake news detection

Supervised text Classification for fake news detection Using Machine Learning Models


Procedure

  • The Dataset
  • Splitting the data into Train and Test data
  • Cleaning the data
  • Building the Document-Term Matrix
  • Model Building
  • Cross validation
  • Making predictions from the model created and displaying a Confusion matrix


Results


Summary of Results

Algorithms Author Package Keyword Accuracy
Kaggle fake news dataset Fake news Detector dataset Gofaaas Fake News Dataset

500 rows

Test data (30% of the dataset) Cross validation Test data (30% of the dataset)
 Cross validation Test data (30% of the dataset) Cross validation Using the Kaggle Model* Using the Detector Model**
Naive Bayes Bayes, Thomas We used RTextTools, which depends on e1071 NB*
Support vector machine Meyer et al., 2012 We used RTextTools, which depends on e1071 SVM*
Random forest Liawand Wiener, 2002 We used RTextTools, which depends on randomForest RF
Extreme Gradient Boosting Chen & Guestrin, 2016 xgboost XGBOOST*
General linearized models Friedman et al., 2010 We used RTextTools, which depends on wglmnet GLMNET*
Maximum entropy Jurka, 2012 We used RTextTools, which depends on maxent MAXENT*
* Accuracy of predictions made with the Kaggle Fake News Model over the Gofaaas Fake New Dataset

** Accuracy of predictions made with the Fake News Detector Model over the Gofaaas Fake New Dataset



Evaluation of Results

We evaluate our approach in different settings. First, weperform cross-validation on our noisy training set; second,and more importantly, we train models on the training setand validate them against a manually created gold standard.17Moreover, we evaluate two variants, i.e., including and exclud-ing user features. [smb:home/adelo/1-system/1-disco_local/1-mis_archivos/1-pe/1-ciencia/1-computacion/2-data_analysis-machine_learning/gofaaaz-machine_learning/5-References/7-Weakly_supervised_searning_for_fake_news_detection_on_twitter.pdf]


The Gofaaas-Fake News Detector R Package


Installation


Functions


Datasets used


Kaggle Fake News Dataset

https://www.kaggle.com/c/fake-news/data


Distribution of the data:

The distribution of Stance classes in train_stances.csv is as follows:

rows unrelated discuss agree disagree
49972 0.73131 0.17828 0.0736012 0.0168094



Fake News Detector Dataset


Gofaaas Fake News Dataset


Algorithms


Naive Bayes

Naïve Bayes is based on the Bayesian theorem, there in order to understand Naïve Bayes it is important to first understand the Bayesian theorem.

Bayesian theorem is a mathematical formula for determining conditional probability which is the probability of something, happening given that something else has already occurred.


Image.png
  • P(c|x) is the posterior probability of class (target) given predictor (attribute).
  • P(c) is the prior probability of class.
  • P(x|c) is the likelihood which is the probability of predictor given class.
  • P(x) is the prior probability of predictor.


Prior probability, in Bayesian statistical inference, is the probability of an event before new data is collected.

Posterior probability is the revised probability of an event occurring after taking into consideration new information.

In statistical terms, the posterior probability is the probability of event A occurring given that event B has occurred.



Support vector machine


Random forest


Extreme Gradient Boosting


The RTextTools package

RTextTools - A Supervised Learning Package for Text Classification:


Chapter 3 - Gofaas Web App

A way to interact, test and display the model results


Conclusion


Retrieved from "http://wiki.sinfronteras.ws/index.php?title=Supervised_Machine_Learning_for_Fake_News_Detection&oldid=8800"